Image processing apparatus, image processing method, and medium on which image processing program is recorded

ABSTRACT

Featured are an image processing apparatus, method and a medium including an image processing program. The image processing apparatus includes an image inputting means for inputting front and back images of an original, an image reversing means for reversing one of the front and back images, a position relationship detection means, image correcting means and image outputting means for outputting the image. The positional relationship detecting means detects the positional relationship between the front image reversed by the image reversing means and the back image from the image inputting means or the positional relationship between the back image reversed by the image reversing means and the front image from the image inputting means. The image correcting means corrects the image to eliminate a ghost image of the image using the positional relationship between the front and back images obtained from the positional relationship detecting means.

This application is a divisional of U.S. application Ser. No.09/720,475, filed Mar. 8, 2001 and now U.S. Pat. No. 7,006,708 B1.

TECHNICAL FIELD

The present invention relates to an image processing apparatus, an imageprocessing method, and a medium on which an image processing program isrecorded. Particularly, the invention relates to an image processingapparatus, an image processing method, and a medium on which an imageprocessing program is recorded, each for converting, to a naturallymonochromatic image, an image which is read by an image inputting meanssuch as a scanner from an original printed in two colors, a color otherthan black, or the like. Further, the invention relates particularly toan image processing apparatus, an image processing method, and a mediumon which an image processing program is recorded, each for contracting acomposite image such as a cartoon comprising a character, a linedrawing, and a halftone dot, with good precision and without a moire.Furthermore, the invention relates particularly to an image processingapparatus, an image processing method, and a medium on which an imageprocessing program is recorded, each for performing the correction toeliminate a ghost image which occurs during the reading of an originalprinted on both sides or an original being stacked. Further, theinvention relates particularly to an image processing apparatus, animage processing method, and a medium on which an image processingprogram is recorded, each for performing the aligning between desiredpages in the image data of a book inputted in an authoring system forpreparing the contents of an electronic book by inputting the book onthe image base.

BACKGROUND ART

The progress in hardware and software has been activating thepublication of electronic books as a new form of books in place ofexisting books on a paper medium. It has been realized to read cartoonsand novels on a personal computer or a portable terminal.

Although these electronic books can be prepared correspondingly to whatis called multimedia data, such as an audio, an image, a dynamic image,and an animation, it is costly and laborious to fabricate an electronicbook as the primary object. Thus, an electronic book is fabricatedfrequently by directly computerizing an existing book.

There are following problems in the fabrication of an electronic book inaccordance with a conventional art. A first problem exists in the caseof converting, to a naturally monochromatic image, an image read by animage inputting means such as a scanner from a color-printed original orthe like.

The conversion from a color image to a monochromatic image is generallyperformed by extracting the brightness component from the color image.The following Formula (1) is a formula for extracting the brightnesscomponent (Y) from red (R), green (G), and blue (B) components.Y=0.299R+0.587G+0.114B   (1)

This method is used also in the NTSC system which is a system for TVbroadcasting, and is widely known. In the NTSC system, a color image isdecomposed into a brightness signal and a color-difference signal andthen transmitted. Therefore, even a monochromatic television set candisplay a naturally monochromatic image on the television screen byreceiving and reproducing the brightness signal alone.

Formula (1) depends on the characteristics of human visual sensation,and a full-color image such as a photograph can be converted to anaturally monochromatic image by converting the brightness component tothe monochromatic image.

Meanwhile, the contents of an electronic book computerized directly froman existing book can be viewed on a color display without any problembecause of the unlimited number of display colors. However, a portableterminal frequently uses a monochromatic liquid-crystal display becauseof the important factors of a low price and a low power consumption.

Accordingly, a full-color image such as a photograph is usuallyconverted to a monochromatic image. Using Formula (1) to convert it to anaturally monochromatic image, monochromatic contents can be producedfrom an existing book. Or, a color image can be displayed on amonochromatic display.

However, in the case of a certain existing book, especially a cartoonjournal, the image is sometimes printed in two colors with red ink andblack ink or in a color with ink of a color selected from the groupconsisting of red, green, blue, and the like other than black. In thatcase, a desirable image quality is hardly obtained by converting such animage using Formula (1).

For example, in the case of an image printed in two colors with red inkand black ink, black is used for contours and shadows, and red is usedfor flesh color. When such an image is converted to a monochromaticimage using Formula (1), the low mixing-ratio of red causes a problemthat the red part becomes darker than the actual image.

Further, an image is sometime printed in a single color other thanblack, such as red, green, and blue. When people view such an image, thecolor itself is not explicitly recognized, and the resulted impressionis similar to that of the image printed in a color of black. However,when such an image is converted to a monochromatic image using Formula(1) in a similar manner to that for a photograph, the resulted image hasa thin color and a low contrast. In particular, an image printed withgreen ink results in an image with a very thin color because of thelarge mixing-ratio of green in Formula (1).

A second problem exists in the case of contracting a composite imagesuch as a cartoon comprising a character, a line drawing, and a halftonedot, with good precision and without a moire.

Printed matter, such as a cartoon and a novel, is originally printed ina very high resolution, and a scanner for reading it also has a highresolution of 400 dpi (dot per inch), 600 dpi, or higher. In contrast,the display resolution of a personal computer or a portable terminal isat most about 100 dpi. Thus, the contraction of an image is necessary todisplay an existing book on a personal computer or a portable terminal.

In many cartoons, a halftone screen is used for the pseudo-expression ofdensity and gradation. Since the halftone screen comprises fine meshdots, lines, and patterns, the contraction thereof, as known, ordinarilyresults in a pattern of stripes or lattice which is called a moire. Inthe invention, a region to which the pseudo-expression of density orgradation is imparted is called a pseudo-density region.

Conventional art of contraction is classified into two major methods: amethod in which the whole region is homogeneously processed and a methodin which the region is divided and each of the divided regions iscontracted optimally. The method in which the whole region ishomogeneously processed generally includes a thinning-out process toachieve the contraction by simply thinning out the pixels and anaveraging technique to determine the pixel value of the contracted imageby averaging the pixel values of the original image (Hitoshi Kiya,Resolution Conversion of Digital Image, CQ Publishing, The Interface,June 1998, p. 72).

With regard to the region-dividing method, an image processing apparatusis described in Japanese Unexamined Patent Publication JP-A 4-365182(1992). In accordance with the image processing apparatus described inJP-A 4-365182, a binary coded image is divided into two regions, i.e.,drawing and painting regions. In the drawing region, contraction isperformed so as to conserve the fine lines. In the painting region, amulti-value encoding process is performed on the base of the pixeldensity, and then a contraction/binary-encoding process is performed,thereby to contract even the character and the drawing sections withprecision and without a moire.

However, in the case of contracting an image such as a cartooncomprising a character, a line drawing; and a halftone dot incombination, the thinning-out process causes a moire in the halftonesection and unclearness and blurring in the character section and theline drawing section. In contrast, in the averaging technique, a moireis suppressed and the unclearness and blurring of a character and a fineline seldom occur, but the clearness of the whole is lost. To expressthe clearness, an edge enhancing process can be used after thecontraction. However, the suppressed moire is also enhanced and appears.Although the moire can be completely eliminated by enlarging the area inwhich the pixels are averaged, characters and drawings blur on the otherhand.

As mentioned above; prior art homogeneous processing by the thinning-outprocess or the averaging process cannot achieve clear characters anddrawings without a moire. Thus, it is necessary to divide the image intoregions and process each region appropriately.

The image processing apparatus described in JP-A 4-365182 performsregion-dividing and performs a contraction process appropriate for eachregion. However, in a cartoon and the like, a line drawing may exist inhalftone dots. Thus, a character and a line drawing cannot be separatedby simple pattern matching. Further, a character exists in a balloonwithin the image. Thus, separation using a simple rectangle isdifficult. In a method wherein an image is separated into two regions,i.e., a character/line drawing region and the other region, and whereinsmoothing is performed in the region other than the character/linedrawing region, an error frequently occurs in the extraction of thecharacter/line drawing region. For example, a rather blurring part in afine line and a complicated character is sometimes not extracted as acharacter/line drawing region because of the low edge component. Theprocess of smoothing of this region causes further blurring in thecharacter and the line drawing section.

A third problem exists in the case of performing the correction toeliminate a ghost image which occurs during the reading of an originalprinted on both sides or an original being stacked.

Since an existing book is ordinarily printed on both sides of a paper,there is a problem of what is called a ghost image, which is an image onthe back side being seen through during the reading thereof with ascanner and the like or during the reading of a page being stacked witha scanner and the like.

An image reader described in Japanese Unexamined Patent Publication JP-A6-14185 (1994) is a prior art apparatus for correcting a ghost image.The image reader described in JP-A 6-14185 eliminates a ghost imagepart, which has a low density, by reducing the density through thedensity correction of the image signal, thereby preventing the copyingof the seen-through image of the back side of an original or the nextpage of a stacked original.

An image forming apparatus described in Japanese Unexamined PatentPublication JP-A 6-62216 (1994) is an apparatus for correcting a ghostimage using front image data and back image data. The image formingapparatus described in JP-A 6-62216 performs an AND operation betweenthe front image data and the back image data, performs the smoothing ofthe output by a histogram calculation, and then performs a thresholdprocess. Then the apparatus combines it with image data which is thefront image data subtracted by the data of the superposed part, therebyeliminating the ghost image without the loss of the low density part ofthe front image.

Further, an image processing apparatus with ghost-image eliminatingfunction described in Japanese Unexamined Patent Publication JP-A8-340447 (1996) eliminates a ghost image by detecting a ghost imageregion in a video signal and the ghost image level within the ghostimage region and by correcting the density of the ghost image regionusing the ghost image level.

However, since the image reader described in Japanese Unexamined PatentPublication JP-A 6-14185 (1994) performs the density correction on thewhole image, there occurs a problem that a halftone section whitens outand that a character becomes unclear.

With regard to the image forming apparatus described in JP-A 6-62216, aghost image cannot be completely eliminated in some cases, for example,in the case where a halftone section is seen through as a ghost image.Further, the positional relationship between the front image and theback image needs to be previously known, but the positions in which theimages are read are not necessarily identical even if automatic paperfeeding is used. Thus, a ghost image cannot be completely eliminated insuch a case where the images are shifted from a predetermined position.

In the image processing apparatus with ghost-image eliminating functiondescribed in JP-A 8-340447, a ghost image region is defined as a regionin which non-character pixels are successive, a region in whichnon-painting pixels are successive, a region in which pixels having adensity lower than or equal to a predetermined density are successive,and a region in which pixels having a saturation lower than or equal toa predetermined saturation are successive. Accordingly, thedetermination is carried out in a small area. Therefore, a ghost imagecannot be separated from a halftone image on the front side, forexample, when the ghost image is caused by a widely spreading black darkregion of characters or images.

A fourth problem exists in the case of performing the aligning betweendesired pages in the image data of a book inputted in an authoringsystem for preparing the contents of an electronic book by inputting thebook on the image base.

Although an electronic book can be prepared correspondingly to what iscalled multimedia data, such as an audio, an image, a dynamic image, andan animation, it adopts a text (character-code) based format. Meanwhile,books on a paper medium, what is called “books” are presently beingpublished at a pace of 500,000 or more titles a year. The accumulatednumber is huge. However, the number of the computerized titles is veryfew and almost all exist only on a paper medium. Such a prior artelectronic book has the following problems because of adopting thetext-(character-code) based format.

For the authoring of a book on a paper medium, the text data needs to beprepared by human work or with an OCR. Thus, the preparation of thecontents requires a lot of time, which causes a difficulty in the timelysupply of contents in a large amount. Further, it is difficult toprepare the contents of a book such as a cartoon and a photographicjournal, in which a majority of the data is non-text. For this reason,present number of the electronic book contents is as small as a fewhundred, and many of the contents are dictionaries. Accordingly, theshare of electronic books is presently below 1% of that of paper books.In particular, the small number of the contents is a fatal problem,which is significantly preventing the spread of electronic books. Insuch a situation, to resolve the above-mentioned problems, it ispossible to obtain electronic book contents by inputting them on theimage base. This has the following advantage.

Contents preparation can be carried out basically only by scanning anexisting book on a paper medium, which permits to supply a large amountof the contents in a short term. It permits to supply the contents of acartoon, a photographic journal, and the like, which was impossible in atext-based electronic book. Inputting is easy even when a book, such asan old document, contains a character not in the present character codesystem, for example, an external character and a heteromorphiccharacter. Overseas deployment (spreading) of a viewer and a totalauthoring system is easy because of the independence of a language(character code). By the advantage mentioned above, an electronic bookon the image base resolves all problems in an electronic book on thetext base. To obtain the electronic book contents by inputting it on theimage base, it can be inputted by scanner inputting with an ADF (autodocument feeder) and the like, and various processes such as documentstructuring are carried out on it. However, the following problemsoccur.

In the case where an image is inclined or shifted during the scannerinputting, an user of the electronic book feels uncomfortablenessbecause the inclination is emphasized more than in the case of a paperbook by a reference line (for example, an edge section of a CRT and aliquid-crystal screen) existing in a viewer. Thus, the process ofcorrecting this is necessary. Manual processing of this needs a lot ofworking time, which causes a substantial increase in the authoring time.In particular, inclination and shift in a main page results in a stronguncomfortableness during the viewing on a viewer. Further, the checkingof all pages for the revision of the electronic book contents causes anincrease in the authoring time, which prevents to supply a large amountof contents in a short term. Thus, an appropriate process is necessary.

A first object of the invention is to provide an image processingapparatus, an image processing method, and a medium on which an imageprocessing program is recorded, each for converting an original printedin two colors or a color other than black to a naturally monochromaticimage.

A second object of the invention is to provide an image processingapparatus, an image processing method, and a medium on which an imageprocessing program is recorded, each for contracting a composite imagesuch as a cartoon comprising a character, a line drawing, and a halftonedot, with clearness and without a moire.

A third object of the invention is to provide an image processingapparatus, an image processing method, and a medium on which an imageprocessing program is recorded, each for correcting an image toeliminate a ghost image which occurs during the reading of an originalprinted on both sides or an original being stacked.

A fourth object of the invention is to provide an image processingapparatus, an image processing method, and a medium on which an imageprocessing program is recorded, each for obtaining aligned images.

DISCLOSURE OF INVENTION

A first invention is an image processing apparatus comprising:

image inputting means;

color analyzing means for analyzing a color used within an input image;

mixing-ratio calculating means for calculating a mixing-ratio of colorcomponents based on an analyzed color; and

converting means for converting the input image to a monochromatic imageby mixing color components according to a calculated mixing-ratio.

In accordance with the invention, the data read from an original imageis inputted from the image inputting means, for example, in everypredetermined unit time. The color analyzing means analyzes the colorused within the input image. Further, the mixing-ratio calculating meanscalculates the mixing-ratio of color components such as red, green andblue, based on the analyzed color. The mixing-ratio is determined so asto correspond to the color used within the input image. The convertingmeans converts the input image to a monochromatic image by mixing colorcomponents according to the calculated mixing-ratio.

Therefore, the color of the input image is automatically determined,whereby a monochromatic image can be produced.

A second invention is characterized in that the image inputting means iscapable of inputting a plurality of images;

the color analyzing means analyzes colors used within the plurality ofinput images;

the mixing-ratio calculating means calculates mixing-ratios of colorcomponents which are common to the plurality of images, based onanalyzed colors; and

the converting means converts the plurality of input images tomonochromatic images by mixing color components according to calculatedmixing-ratios.

In accordance with the invention, the data read from the plurality oforiginal images is inputted from the image inputting means, for example,in every predetermined unit time. The color analyzing means analyzescolors used within the plurality of input images. Further, themixing-ratio calculating means calculates mixing-ratios of colorcomponents which are common to the plurality of images, based on theanalyzed colors. That is, the mixing-ratios are determined to correspondto the colors used within the plurality of input images. The convertingmeans converts the plurality of input images to monochromatic images bymixing color components according to the calculated mixing-ratios.

Therefore, the colors of the plurality of input images are automaticallydetermined, whereby monochromatic images can be produced. Further, sincecolor determination is carried out with respect to the plurality ofinput images, more accurate determination can be achieved. Furthermore,since monochromatic images are produced in the same conditions for theplurality of input images, the images can be produced stably.

A third invention is an image processing apparatus comprising:

image inputting means;

color specifying means for externally specifying a color used within aninput image;

mixing-ratio calculating means for calculating a mixing-ratio of colorcomponents based on a specified color; and

converting means for converting the input image to a monochromatic imageby mixing color components according to a calculated mixing-ratio.

In accordance with the invention, the color specifying means specifies acolor used within the image inputted from the image inputting means.Further, the mixing-ratio calculating means calculates the mixing-ratioof color components based on the specified color. That is, themixing-ratio is determined so as to correspond to the color used withinthe input image. The converting means converts the input image to amonochromatic image by mixing color components according to thecalculated mixing-ratio.

Therefore, a monochromatic image with higher accuracy can be produced byspecifying the color used within the input image by a user.

A fourth invention is an image processing apparatus comprising:

image inputting means;

mixing-ratio specifying means for externally specifying a mixing-ratioof color components; and

converting means for converting an input image to a monochromatic imageby mixing color components according to a specified mixing-ratio.

In accordance with the invention, the mixing-ratio specifying meansspecifies the mixing-ratio of color components for the image inputtedfrom the image inputting means. The converting means converts the inputimage to a monochromatic image by mixing color components according tothe specified mixing-ratio.

Therefore, a desired monochromatic image can be produced by specifyingthe mixing-ratio of color components by a user.

A fifth invention is an image processing method comprising:

a color analyzing step of analyzing a color used within an input image;

a mixing-ratio calculating step of calculating a mixing-ratio of colorcomponents based on an analyzed color; and

a converting step of converting the input image to a monochromatic imageby mixing color components according to a calculated mixing-ratio.

In accordance with the invention, an original image is read, forexample, in every predetermined unit time, and the data is inputted. Thecolor used within the image is analyzed. The mixing-ratio of colorcomponents is calculated based on the analyzed color, whereby themixing-ratio is determined so as to correspond to the color used withinthe input image. Further, the color components are mixed according tothe calculated mixing-ratio, and the input image is converted to amonochromatic image.

Therefore, the color of the input image is automatically determined,whereby a monochromatic image can be produced.

A sixth invention is characterized in that in the color analyzing stepcolor analysis is carried out based on distribution of hue, saturationand lightness of the input image.

In accordance with the invention, analysis of the color used within theimage is carried out, based on the distribution of hue, saturation andlightness of the input image. For example, representative hue anddispersion are calculated from the distribution of hue and saturation.Then, the presence or absence of black pixels is determined from thehistogram of lightness. Analysis of the colors used within the inputimage is further carried out. That is, when representative hue isabsent, the image is determined as a monochromatic image. When thedispersion is greater than or equal to a predetermined value, the imageis determined as an image in which plural colors are used. In the casewhere representative hue is absent and the dispersion is greater than orequal to a predetermined value, the image is determined as amonochromatic image in a color other than black when a black pixel isabsent, and the image is determined as an image in black plus a colorother than black when a black pixel is present. Further, themixing-ratio of color components is calculated based on the analyzedcolors, whereby the mixing-ratio is determined according to the colorsused within the input image. Further, the color components are mixedaccording to the calculated mixing-ratio, and the input image isconverted to a monochromatic image.

Therefore, it is possible to produce a monochromatic image byautomatically determining the color of the input image.

A seventh invention is an image processing method comprising:

a color analyzing step of analyzing colors used within a plurality ofinput images;

a mixing-ratio calculating step of calculating mixing-ratios of colorcomponents which are common to the plurality of input images, based onanalyzed colors; and

a converting step of converting the plurality of input images tomonochromatic images by mixing color components according to calculatedmixing-ratios.

In accordance with the invention, data is inputted, for example, byreading a plurality of original images in every predetermined unit time,colors used within the images are analyzed, and the mixing-ratios ofcolor components which are common to the plurality of images arecalculated according to the analyzed colors, whereby the mixing-ratiosare determined in correspondence to the colors used within the inputimages. Further, the color components are mixed according to thecalculated mixing-ratios, and the plurality of input images areconverted to monochromatic images.

Therefore, the colors of the plurality of input images are automaticallydetermined and monochromatic images can be produced. Further, since thecolors are determined from the plurality of input images, the colors canbe determined more accurately. Furthermore, since monochromatic imagesare produced in the same conditions for the plurality of input images,the images can be produced stably.

An eighth invention is an image processing method comprising:

a color specifying step of externally specifying a color used within aninput image;

a mixing-ratio calculating step of calculating a mixing-ratio of colorcomponents based on a specified color; and

a converting step of converting the input image to a monochromatic imageby mixing color components according to a calculated mixing-ratio.

In accordance with the invention, the color used within the input imageis specified, the mixing-ratio of color components is calculated basedon the color, the mixing-ratio is determined correspondingly to thecolor used within the input image, color components are mixed accordingto the calculated mixing-ratio, and the input image is converted to amonochromatic image.

Therefore, a user can produce a monochromatic image with higher accuracyby specifying the color used within the input images.

A ninth invention is characterized in that in the mixing-ratiocalculating step a mixing-ratio is calculated based on a mixing-ratiotable in which a mixing-ratio of color components corresponding to thecolor used within the input image is previously stored.

In accordance with the invention, the color used within an input imageis analyzed or specified. The mixing-ratio of color components iscalculated based on the color used within the input image. Themixing-ratio is calculated by referring to the mixing-ratio table.Further, color components are mixed according to the calculatedmixing-ratio, and the input image is converted to a monochromatic image.

Therefore, it is possible to produced monochromatic image byautomatically determining the color of the input image. Further, bycalculating the mixing-ratio with reference to the mixing-ratio table,an optimum mixing-ratio is rapidly obtained for each color used withinthe image. Thus, a more optimum monochromatic image can be produced athigh speed.

A tenth invention is characterized in that in the mixing-ratiocalculating step the mixing-ratio is calculated based on a ratio ofcolor components of a complimentary color of the color used within theinput image.

In accordance with the invention, the color used within an input imageis analyzed or specified. The mixing-ratio of color components iscalculated based on the color used within the input image. Here, themixing-ratio is calculated based on the color component ratio of acomplimentary color of the color used within the input image. Further,the color components are mixed according to the calculated mixing-ratio,and the input image is converted to a monochromatic image.

Therefore, it is possible to produce a monochromatic image with highcontrast by automatically determining the colors of the input image.

An eleventh invention is characterized in that in the mixing-ratiocalculating step the mixing-ratio is calculated based on a colorcomponent ratio of a complimentary color of the color used within theinput image and the color component ratio of the colors used within theinput image.

In accordance with the invention, the color used within an input imageis analyzed or specified. The mixing-ratio of color components iscalculated based on the color, and accordingly the mixing-ratio isdetermined correspondingly to the color used within the input image. Themixing-ratio is calculated based on the color component ratio of thecomplimentary color of the color used within the input image and thecolor component ratio of the color used within the input image. Further,the color components are mixed according to the calculated mixing-ratio,and the input image is converted to a monochromatic image.

Therefore, the color of the input image is automatically determined, anda high-contrast monochromatic image in which discrimination between thecolor used in the image and black is easily carried out can be produced.

A twelfth invention is an image processing method comprising:

a mixing-ratio specifying step of externally specifying a mixing-ratioof color components of an input image; and

a converting step of converting the input image to a monochromatic imageby mixing color components according to a specified mixing-ratio.

In accordance with the invention, the mixing-ratio of color componentsof the input image is specified, and by mixing color componentsaccording to the mixing-ratio, the input image is converted to amonochromatic image.

Therefore, a desired monochromatic image can be produced by specifyingthe mixing-ratio of color components.

A thirteenth invention is an medium on which an image processing programis recorded, the image processing program being for causing a computerto execute a color analyzing step of analyzing a color used within aninput image; a mixing-ratio calculating step of calculating amixing-ratio of color components based on a analyzed color; and aconverting step of converting the input image to a monochromatic imageby mixing color components according to a calculated mixing-ratio.

In accordance with the invention, by following the image processingprogram recorded on the medium, a computer analyzes the color usedwithin an input image, calculates the mixing-ratio of color componentsbased on the color thereby to determine the mixing-ratio correspondinglyto the color used within the input image, mixes the color componentsaccording to the mixing-ratio, and achieves the conversion from theinput image to a monochromatic image. Therefore, the color of the inputimage is automatically determined and a monochromatic image can beproduced.

A fourteenth invention is an medium on which an image processing programis recorded, the image processing program being for causing a computerto execute a color analyzing step of analyzing colors used within aplurality of input images; a mixing-ratio calculating step ofcalculating mixing-ratios of color components which are common to theplurality of input images, based on analyzed colors; and a convertingstep of converting the plurality of input images to monochromatic imagesby mixing color components according to calculated mixing-ratios.

In accordance with the invention, by following the image processingprogram recorded on the medium, a computer analyzes the color usedwithin a plurality of input images, calculates the mixing-ratios ofcolor components which are common to the plurality of input images,based on the colors, thereby to determine the mixing-ratioscorrespondingly to the colors used within the plurality of input images,mixes color components according to the mixing-ratios, and achieves theconversion from the plurality of input images to monochromatic images.Therefore, it is possible to produce monochromatic images byautomatically determining the colors of the plurality of input images.

A fifteenth invention is an medium on which an image processing programis recorded, the image processing program being for causing a computerto execute a color specifying step of externally specifying a color usedwithin an input image; a mixing-ratio calculating step of calculating amixing-ratio of color components based on a specified color; and aconverting step of converting the input image to a monochromatic imageby mixing color components according to a calculated mixing-ratio.

In accordance with the invention, by following the image processingprogram recorded on the medium, a computer specifies a color used withinan input image, calculates the mixing-ratio of color components based onthe color thereby to determine the mixing-ratio correspondingly to thecolor used within the input image, mixes color components according tothe mixing-ratio, and achieves the conversion from the input image to amonochromatic image. Therefore, it is possible to produce amonochromatic image with higher accuracy by specifying the color usedwithin the input image by a user.

A sixteenth invention is an medium on which an image processing programis recorded, the image processing program being for causing a computerto execute a mixing-ratio specifying step of externally specifying amixing-ratio of color components of an input image; and a convertingstep of converting the input image to a monochromatic image by mixingcolor components according to a specified mixing-ratio.

In accordance with the invention, by following the image processingprogram recorded on the medium, a computer specifies a mixing-ratio foran input image, mixes the color components according to the specifiedmixing-ratio, and achieves the conversion from the input image to amonochromatic image. Therefore, it is possible to produce a desiredmonochromatic image by specifying the mixing-ratio of color componentsby a user.

A seventeenth invention is an image processing apparatus comprising:

image inputting means for inputting an image;

character/line drawing region extracting means for extracting acharacter/line drawing region from the input image;

pseudo-density region extracting means for extracting a pseudo-densityregion from the input image;

image contracting means for contracting images in an extractedpseudo-density region, an extracted character/line drawing region, and aregion other than the pseudo-density region and the character/linedrawing region, by mutually different methods; and

image outputting means for outputting the contracted image.

In accordance with the invention, an image is inputted by the imageinputting means, and from the image a character/line drawing region isextracted by the character/line drawing region extracting means and apseudo-density region is extracted by the pseudo-density regionextracting means. The image contracting means contracts the image usingmutually different methods respectively in the pseudo-density region,the character/line drawing region, and the other region. The imageoutputting means outputs the contracted image.

Therefore, by dividing the input image into three regions, i.e., apseudo-density region, a character/line drawing region and the otherregion, and by contracting the image in each region using a differentmethod, the image can be contracted with a moire being suppressed in thepseudo-density region, the image can be clearly contracted in thecharacter/line drawing region, and the image can be properly contractedin the other region. For example, even in the case where a moire occursin the contraction of an image read and inputted at a predeterminedresolution, the image read and inputted at a predetermined resolutioncan be clearly contracted without the occurrence of a moire inaccordance with the invention.

An eighteenth invention is characterized in that the image contractingmeans performs a smoothing process in the pseudo-density region,performs an averaging process and a subsequent edge enhancing process inthe character/line drawing region, and performs an averaging process ina region other than the pseudo-density region and the character/linedrawing region.

In accordance with the invention, an image is inputted by the imageinputting means, and from the image a character/line drawing region isextracted by the character/line drawing region extracting means and apseudo-density region is extracted by the pseudo-density regionextracting means. The image contracting means contracts the image usingmutually different methods respectively in the pseudo-density region,the character/line drawing region and the other region. The image iscontracted, by a smoothing process in the pseudo-density region, by anaveraging process and a subsequent edge enhancing process in thecharacter/line drawing region, and by an averaging process in the otherregion. The image outputting means outputs the contracted image.

Therefore, the image can be contracted with a moire being suppressed inthe pseudo-density region, the image can be clearly contracted in thecharacter/line drawing region, and the image can be properly contractedin the other region.

A nineteenth invention is characterized in that the character/linedrawing region extracting means extracts a character/line drawing regionfrom the input image before the extraction of a pseudo-density region.

In accordance with the invention, an image is inputted by the imageinputting means, and from the image the character/line drawing regionextracting means extracts a character/line drawing region and thereafterthe pseudo-density region extracting means extracts a pseudo-densityregion. The image contracting means contracts the image using mutuallydifferent methods respectively in the pseudo-density region, thecharacter/line drawing region and the other region. The image outputtingmeans outputs the contracted image.

Therefore, from the input image a character/line drawing region isfirstly extracted and a pseudo-density region is then extracted.Therefore, the character/line drawing region can be accurately extractedwithout being affected from the pseudo-density region, even when itexists within the pseudo-density region.

A twentieth invention is characterized in that the character/linedrawing region extracting means extracts a character/line drawing regionby performing an edge extraction of the input image after performing asmoothing process thereof.

In accordance with the invention, an image is inputted by the imageinputting means, and from the image the character/line drawing regionextracting means extracts a character/line drawing region and thereafterthe pseudo-density region extracting means extracts a pseudo-densityregion. The character/line drawing region is extracted by the edgeextraction of the input image after the smoothing process thereof. Theimage contracting means contracts the image using mutually differentmethods respectively in the pseudo-density region, the character/linedrawing region and the other region. The image outputting means outputsthe contracted image.

Therefore, from the input image a character/line drawing region isfirstly extracted as mentioned above and a pseudo-density region is thenextracted. Therefore, the character/line drawing region can beaccurately extracted without being affected from the pseudo-densityregion, even when it exists within the pseudo-density region.

A twenty-first invention is characterized in that the pseudo-densityregion extracting means calculates a dispersion of peripheral pixelsaround each pixel of the input image and extracts, as a pseudo-densityregion, the pixel which is one of pixels having a large dispersion andexists in a region which is not extracted as a character/line drawingregion by the character/line drawing region extracting means.

In accordance with the invention, an image is inputted from the imageinputting means, and from the input image the character/line drawingregion extracting means extracts a character/line drawing region and thepseudo-density region extracting means extracts a pseudo-density region.From the input image the character/line drawing region is firstlyextracted and thereafter the pseudo-density region is extracted.Further, the character/line drawing region is extracted by thepredetermined technique described above. The pseudo-density region isextracted by calculating the dispersion of the peripheral pixel's aroundeach pixel of the input image and by extracting, as a pseudo-densityregion, the pixel which is one of the pixels having a large dispersionand exists in the region which is not extracted as a character/linedrawing region. The image contracting means contracts the image usingmutually different methods respectively in the pseudo-density region,the character/line drawing region and the other region. The imageoutputting means outputs the contracted image.

Therefore, by calculating the dispersion of peripheral pixels and byextracting, as a pseudo-density region, the pixel is one of the pixelshaving a large dispersion and exists in the region which is notextracted as a character/line drawing region, the character/line drawingregion is eliminated, whereby the pseudo-density region alone can beextracted accurately.

A twenty-second invention is characterized in that the pseudo-densityregion extracting means calculates a correlation of peripheral pixelsaround each pixel of the input image and extracts, as a pseudo-densityregion, a pixel which is one of pixels having a low correlation andexists in the region which is not extracted as a character/line drawingregion by the character/line drawing region extracting means.

In accordance with the invention, an image is inputted from the imageinputting means, and from the image the character/line drawing regionextracting means extracts a character/line drawing region and thepseudo-density region extracting means extracts a pseudo-density region.From the input image the character/line drawing region is firstlyextracted and thereafter the pseudo-density region is extracted.Further, the character/line drawing region is extracted by thepredetermined technique described above. The pseudo-density region isextracted by calculating the correlation of the peripheral pixels aroundeach pixel of the input image and by extracting, as a pseudo-densityregion, the pixel which is one of the pixels having a low correlationand exists in the region which is not extracted as a character/linedrawing region. The image contracting means contracts the image usingmutually different methods respectively in the pseudo-density region,the character/line drawing region and the other region. The imageoutputting means outputs the contracted image.

Therefore, by calculating the correlation of peripheral pixels and byextracting, as a pseudo-density region, the pixel which is one of thepixels having a low correlation and exists in the region which is notextracted as a character/line drawing region, the character/line drawingregion is eliminated more securely, whereby the pseudo-density regionalone can be extracted accurately.

A twenty-third invention is characterized in that the pseudo-densityregion extracting means detects an edge region of the input image andextracts, as a pseudo-density region, a region which is one of theextracted edge regions and is not extracted as a character/line drawingregion by the character/line drawing region extracting means.

In accordance with the invention, an image is inputted from the imageinputting means, and from the image the character/line drawing regionextracting means extracts a character/line drawing region and thepseudo-density region extracting means extracts a pseudo-density region.From the input image the character/line drawing region is firstlyextracted from the input image and thereafter the pseudo-density regionis extracted. Further, the character/line drawing region is extracted bythe predetermined technique described above. Here, the pseudo-densityregion is extracted by detecting an edge region of the input image andby extracting, as a pseudo-density region, the region which is one ofthe edge regions and is not extracted as a character/line drawingregion. The image contracting means contracts the image using mutuallydifferent methods respectively in the pseudo-density region, thecharacter/line drawing region and the other region. The image outputtingmeans outputs the contracted image.

Therefore, the edge filter is simple, and the pseudo-density region canbe extracted faster.

A twenty-fourth invention is characterized in that contracting meansperforms edge detection of an extracted pseudo-density region andrepeats the smoothing process for a region having a density greater thanor equal to a predetermined value.

In accordance with the invention, an image is inputted from the imageinputting means, and from the image the character/line drawing regionextracting means extracts a character/line drawing region and thepseudo-density region extracting means extracts a pseudo-density region.The image contracting means contracts the image by the smoothing processin the pseudo-density region, contracts the image by the averagingprocess and a subsequent edge enhancing process in the character/linedrawing region, and contracts the image by an averaging process in theother region. The edge detection is carried out in the pseudo-densityregion and the smoothing process is repeated for a region having adensity greater than or equal to a predetermined value. The imageoutputting means outputs the contracted image.

Therefore, occurrence of moire can be securely suppressed in thepseudo-density region and the image can be precisely contracted.

A twenty-fifth invention is characterized in that the image contractingmeans performs edge detection of the extracted pseudo-density region andinterrupts a contracting process for a region having a density greaterthan or equal to a predetermined value.

In accordance with the invention, an image is inputted from the imageinputting means, and from the image the character/line drawing regionextracting means extracts a character/line drawing region and thepseudo-density region extracting means extracts a pseudo-density region.The image contracting means contracts the image using mutually differentmethods respectively in the pseudo-density region, the character/linedrawing region and the other region. Here, the edge detection isperformed for the pseudo-density region and for a region having adensity greater than or equal to a predetermined value, the contractingprocess is interrupted.

Therefore, the normal contracting process can be continued without anunnecessary contracting process.

A twenty-sixth invention is an image processing method comprising:

an image inputting step;

a character/line drawing region extracting step of extracting acharacter/line drawing region from an input image;

a pseudo-density region extracting step of extracting a pseudo-densityregion from the input image;

a image contracting step of contracting the image using mutuallydifferent methods respectively in the extracted pseudo-density region,the extracted character/line drawing region and the region other thanthe pseudo-density region and the character/line drawing region; and

an image outputting step of outputting an contracted image.

In accordance with the invention, an image is inputted, and from theimage a character/line drawing region is extracted and a pseudo-densityregion is extracted. The image is contracted using mutually differentmethods respectively in the pseudo-density region, the character/linedrawing region and the other region. The contracted image is thenoutputted. Therefore, the image can be contracted with a moire beingsuppressed in the pseudo-density region, the image can be clearlycontracted in the character/line drawing region, and the image can beproperly contracted in the other region.

A twenty-seventh invention is an medium recording an image processingprogram for causing a computer to execute an image inputting step; acharacter/line drawing region extracting step of extracting acharacter/line drawing region from an input image; a pseudo-densityregion extracting step of extracting a pseudo-density region from theinput image; an image contracting step of contracting the image usingmutually different methods respectively in the extracted pseudo-densityregion, the extracted character/line drawing region and the region otherthan the pseudo-density region and the character/line drawing region;and an image outputting step of outputting a contracted image.

In accordance with the invention, by following the image processingprogram recorded on the medium, a computer extracts a character/linedrawing region from the image, extracts a pseudo-density region,contracts the image using mutually different methods respectively in thepseudo-density region, the character/line drawing region, and the otherregion, and outputs it. Therefore, the image can be contracted with amoire being suppressed in the pseudo-density region, the image can beclearly contracted in the character/line drawing region, and the imagecan be properly contracted in the other region.

A twenty-eighth invention is an image processing apparatus comprising:

image inputting means for inputting front and back images of anoriginal;

image reversing means for reversing one of the front and back images;

positional relationship detecting means for detecting the positionalrelationship between the front image reversed by the image reversingmeans and the back image from the image inputting means or thepositional relationship between the back image reversed by the imagereversing means and the front image from the image inputting means;

image correcting means for correcting the image to eliminate a ghostimage of the image using the positional relationship between the frontand back images obtained from the positional relationship detectingmeans; and

image outputting means for outputting the image.

In accordance with the invention, front and back images are inputtedfrom the image inputting means. After one of the images is reversed bythe image reversing means, the positional relationship between the frontand back images is detected by the positional relationship detectingmeans. The image is corrected to be free from a ghost image by the imagecorrecting means using the positional relationship, and then outputtedby the image outputting means. Therefore, the input image can beoutputted without a ghost image.

A twenty-ninth invention is characterized in that the positionalrelationship detecting means detects the positional relationship betweenthe front and back images by extracting the high brightness componentalone of the front and back images and by performing the block matchingof the high brightness component.

In accordance with the invention, front and back images are inputtedfrom the image inputting means. After any one of the images is reversedby the image reversing means, the positional relationship between thefront and back images is detected by the positional relationshipdetecting means. Here, the detection of the positional relationshipbetween the front and back images is carried out by extracting the highbrightness component alone of the front and back images and byperforming the block matching of the high brightness component. Theimage is corrected to be free from a ghost image by the image correctingmeans using the positional relationship, and then outputted by the imageoutputting means. Accordingly, the positional relationship can bedetected more precisely, and the input image can be outputted moresecurely without a ghost image.

A thirtieth invention is an image processing apparatus comprising:

image inputting means;

edge detecting means for detecting an edge of the image form the imageinputting means;

image correcting means for correcting the image to eliminate a ghostimage of the image by raising the brightness of high brightness pixelsother than the edge of the image outputted from the edge detectingmeans; and

image outputting means for outputting the image.

In accordance with the invention, the image inputting means inputs animage, and the edge detecting means detects an edge of the image. Theimage correcting means corrects the image to eliminate a ghost image ofthe image by raising the brightness of high brightness pixels other thanthe edge of the image outputted from the edge detecting means. The imageoutputting means outputs the image. Accordingly, the input image can beoutputted with the unclearness of a character being prevented andwithout a ghost image.

A thirty-first invention is an image processing apparatus comprising:

image inputting means;

edge detecting means for detecting an edge of the image form the imageinputting means;

image dividing means for dividing the image depending on the edge andlow brightness pixels of the image outputted from the edge detectingmeans;

image correcting means for correcting the image to eliminate a ghostimage of the image by calculating the average brightness within a regiondivided by the image dividing means and by raising the brightness of thehigh brightness region alone; and

image outputting means for outputting the image.

In accordance with the invention, the image inputting means inputs animage, and the edge detecting means detects an edge of the image. Theimage dividing means divides the image based on the edge and lowbrightness pixels of the image outputted from the edge detecting means.The image correcting means corrects the image to eliminate a ghost imageof the image by calculating the average brightness within a dividedregion and by raising the brightness of the high brightness regionalone. The image outputting means outputs the image. Accordingly, theinput image can be outputted with the whitening-out of a halftonesection being prevented and without a ghost image.

A thirty-second invention is characterized in that the image correctingmeans acquires an representative brightness from the pixels having abrightness within a predetermined range, thereby raising the brightnessof the region with referencing to the representative brightness.

In accordance with the invention, the image inputting means inputs animage, and the edge detecting means detects an edge of the image. Theimage dividing means divides the image based on the edge and the lowbrightness pixel of the image outputted from the edge detecting means.The image correcting means corrects the image to eliminate a ghost imageof the image by calculating the average brightness within a dividedregion and by raising the brightness of the high brightness regionalone. Here, it acquires an representative brightness from the pixelshaving a brightness within a predetermined range, thereby raising thebrightness of the region with referencing to the representativebrightness. The image outputting means outputs the corrected image.Accordingly, the input image can be outputted without a ghost image,free from the influence of the difference in the transmittance dependingon the paper quality.

A thirty-third invention is an image processing method comprising:

an image reversing step of reversing one of front and back images of anoriginal;

a positional relationship detecting step of detecting a positionalrelationship between the reversed one and the other of the front andback images; and

an image correcting step of correcting the other one to eliminate aghost image of the reversed one using a result of the positionalrelationship detection.

In accordance with the invention, front and back images are inputted,and after one of the images is reversed, the positional relationshipbetween the reversed one and the other one of the front and back imagesis detected, the other one is corrected to be free from a ghost image ofthe reversed one using the positional relationship, and then outputted.Accordingly, the input image can be outputted without a ghost image.

A thirty-fourth invention is an image processing method comprising:

an image-edge detecting step of detecting an edge of an image; and

an image correcting step of correcting the image to eliminate a ghostimage from the image by raising the brightness of high brightness pixelsother than a detected edge.

In accordance with the invention, an image is inputted, an edge of theimage is detected to correct the image to be free from a ghost image inthe image by raising the brightness of high brightness pixels other thanthe edge of the image outputted from the edge detection and output theresulting image. Accordingly, it is possible to output the input imagewithout unclearness of characters and a ghost image.

A thirty-fifth invention is an image processing method comprising:

an image-edge detecting step of detecting an edge of an image;

an image dividing step of dividing the image based on a detected, edgeand low brightness pixels; and

an image correcting step of correcting the image to eliminate a ghostimage from the image by calculating an average brightness within adivided region and by raising a brightness of the high brightness regionalone.

In accordance with the invention, an image is inputted, an edge of theimage is detected, the image is divided based on the edge and the lowbrightness pixels of the image outputted from the edge detection, andthe image is corrected to be free from a ghost image by calculating theaverage brightness within a divided region and by raising the brightnessof the high brightness region alone, and then outputted. Accordingly, itis possible to output the input image without whitening-out of ahalftone section and a ghost image.

A thirty-sixth invention is an medium on which an image processingprogram is recorded, the image processing program being for causing acomputer to execute an image reversing step of reversing one of frontand back images; a positional relationship detecting step of detecting apositional relationship between the reversed one and the other of thefront and back images; and an image correcting step of correcting theimage to eliminate a ghost image from the other using a result of thepositional relationship detection.

In accordance with the invention, by following the image processingprogram recorded on the medium, a computer reverses one of inputtedfront and back images, thereafter, detects the positional relationshipbetween the reversed one and the other of the front and back images,corrects the image to eliminate a ghost image from the other image usingthe positional relationship, and outputs a resulting image. Accordingly,the input image can be outputted without a ghost image.

A thirty-seventh invention is an medium on which an image processingprogram is recorded, the image processing program being for causing acomputer to execute an image-edge detecting step of detecting an edge ofan image; and an image correcting step of correcting the image toeliminate a ghost image of the image by raising a brightness of a highbrightness pixel other than the detected edge.

In accordance with the invention, by following the image processingprogram recorded on the medium, a computer detects an edge of aninputted image, corrects the image to eliminate a ghost image from theimage by raising the brightness of a high brightness pixel other thanthe edge of the image outputted from the edge detection, and outputs aresulting image. Accordingly, the input image can be outputted withoutunclearness of characters and a ghost image.

A thirty-eighth invention is an medium on which an image processingprogram is recorded, the image processing program being for causing acomputer to execute an image-edge detecting step of detecting an edge ofan image; an image dividing step of dividing the image based on adetected edge and low brightness pixels; and an image correcting step ofcorrecting the image to eliminate a ghost image from the image bycalculating an average brightness within a divided region and by raisinga brightness of a high brightness region alone.

In accordance with the invention, by following the image processingprogram recorded on the medium, a computer detects an edge of aninputted image, divides the image based on the edge and the lowbrightness pixels of the image outputted from the edge detection,corrects the image to eliminate a ghost image from the image bycalculating the average brightness within a divided region and byraising the brightness of the high brightness region alone, and outputsa resulting image. Accordingly, it is possible to output the input imagewithout a ghost image with the whitening-out of a halftone section beingprevented.

A thirty-ninth invention is an image processing apparatus comprising:

image inputting means for inputting an image page by page;

image determining means for determining a predetermined image from amonginputted images;

template acquiring means for acquiring a template used as an alignmentreference from an image which is determined as the predetermined image;and

image correcting means for correcting a position between the imagesbased on the template, thereby aligning images of consecutive pages.

In accordance with the invention, the image inputting means inputs animage page by page, the image determining means determines apredetermined image from among the images, the template acquiring meansacquires a template from the determined image, and the image correctingmeans corrects the position between the images based on the template,whereby the images of consecutive pages are aligned.

Accordingly, the alignment between desired consecutive images from amongthe images inputted page by page can be carried out in a short time.

A fortieth invention is an image processing apparatus comprising:

image inputting means for inputting an image page by page of a book;

image determining means for determining a predetermined main-text imagefrom among inputted images;

template acquiring means for acquiring a template used as an alignmentreference from an image which is determined as the predeterminedmain-text image; and

image correcting means for correcting a position between the main-textimages based on the template, thereby aligning the main-text images ofconsecutive pages.

In accordance with the invention, the image inputting means inputs animage page by page of a book, the image determining means determines apredetermined main-text image from among the images, the templateacquiring means acquires a template from the determined image, and theimage correcting means corrects the position between main-text imagesbased on the template, whereby the main-text images of consecutive pagesare aligned.

Accordingly, the alignment between main-text images from among themain-text images inputted page by page can be carried out in a shorttime. Thus, the contents of an electronic book can be prepared in ashort term. Further, since the position of the main-text images isaligned when the electronic book is viewed in a viewer,uncomfortableness to a user can be eliminated.

A forty-first invention is an image processing method comprising:

an image determining step of determining a predetermined image fromamong images inputted page by page;

a template acquiring step of acquiring a template used as an alignmentreference from an image which is determined as the predetermined image;and

an image correcting means for correcting a position between the imagesbased on the template, thereby aligning images of consecutive pages.

In accordance with the invention, a predetermined image is determinedfrom among the images inputted page by page. A template is acquired fromthe determined image. The position between the images is corrected basedon the template, thereby aligning the images of consecutive pages.Accordingly, the alignment between desired consecutive images from amongthe images inputted page by page can be carried out in a short time.

A forty-second invention is characterized in that the template acquiringstep is a step of acquiring, as a template, positional information of arectangle defined by circumscribing lines obtained from an ensemble ofedge points acquired by scanning the input image.

In accordance with the invention, a predetermined image is determinedfrom among the images inputted page by page, and a template is acquiredfrom the determined image. Here, the template is acquired as thepositional information of the rectangle defined by the circumscribinglines obtained from the ensemble of the edge points acquired by scanningthe input image. The position between the images is corrected based onthe template, thereby aligning the images of consecutive pages.Accordingly, since the template is acquired using the circumscribinglines, an accurate template can be obtained, thereby improving theprecision of the alignment.

A forty-third invention is characterized in that the image processingmethod further comprises a step of generating warning data in the casewhere the predetermined image is determined from among input imagesduring the image determining step and that positional information of theinput image and positional information of the template are out of apredetermined range.

In accordance with the invention, a predetermined image is determinedfrom among the images inputted page by page, and a template is acquiredfrom the determined image, as described above. The position between theimages is corrected based on the template, thereby aligning the imagesof consecutive pages. Warning data is generated in the case where thepredetermined image is determined from among the input images and thepositional information of the input image and the positional informationof the template are out of a predetermined range. Accordingly, failurein the alignment between the images can be detected, and hence, there isconvenience in revision during or after the authoring.

A forty-fourth invention is an image processing method comprising:

an image determining step of determining a predetermined main-text imagefrom among images inputted page by page of a book;

a template acquiring step of acquiring a template used as an alignmentreference from an image which is determined as the predeterminedmain-text image; and

an image correcting step of correcting a position between the main-textimages based on the template, thereby aligning main-text images ofconsecutive pages.

In accordance with the invention, a predetermined main-text image isdetermined from among the images inputted page by page of a book. Atemplate is acquired from the determined image. The position betweenmain-text images is corrected based on the template, thereby aligningthe main-text images of consecutive pages.

Accordingly, the alignment between main-text images from among themain-text images inputted page by page can be carried out in a shorttime. Thus, the contents of an electronic book can be prepared in ashort term. Further, since the position of the main-text images isaligned when the electronic book is viewed in a viewer,uncomfortableness to a user can be eliminated.

A forty-fifth invention is an medium on which an image processingprogram is recorded, the image processing program being for causing acomputer to execute an image determining step of deter mining apredetermined image from among images inputted page by page; a templateacquiring step of acquiring a template used as an alignment referencefrom an image which is determined as the predetermined image; and imagecorrecting means for correcting a position between images based on thetemplate, thereby aligning images of consecutive pages.

In accordance with the invention, by following the image processingprogram recorded on the medium, a computer determines a predeterminedimage from among the images inputted page by page, acquires a templatefrom the determined image, and corrects the position between the imagesbased on the template, thereby aligning the images of consecutive pages.Accordingly, the alignment between desired consecutive images from amongthe images inputted page by page can be carried out in a short time.

A forty-sixth invention is an medium on which an image processingprogram is recorded, the image processing program being for causing acomputer to execute an image determining step of determining apredetermined main-text image from among images inputted page by page ofa book; template acquiring step of acquiring a template used as analignment reference from an image which is determined as thepredetermined main-text image; and image correcting step of correcting aposition between the main-text images based on the template, therebyaligning main-text images of consecutive pages.

In accordance with the invention, by following the image processingprogram recorded on the medium, a computer determines a predeterminedmain-text image from among the images inputted page by page of a book,acquires a template from the determined image, and corrects the positionbetween main-text images based on the template, thereby aligning themain-text images of consecutive pages. Accordingly, the alignmentbetween main-text images from among the main-text images inputted pageby page can be carried out in a short time. Thus, the contents of anelectronic book can be prepared in a short term. Further, since theposition of the main-text images is aligned when the electronic book isviewed in a viewer, uncomfortableness to a user can be eliminated.

BRIEF DESCRIPTION OF THE DRAWINGS

Other and further objects, features, and advantages of the inventionwill be more explicit from the following detailed description taken withreference to the drawings wherein:

FIG. 1 is a block diagram of an image processing apparatus 1 a inaccordance with a first embodiment of the invention.

FIG. 2 is a flow chart showing the image processing method of the imageprocessing apparatus 1 a.

FIG. 3 is a flow chart for describing a color analyzing section 3.

FIG. 4 is a graph for describing a step S22.

FIG. 5 is a graph for describing a step S23.

FIG. 6 is a graph for describing a step S24.

FIG. 7 is a block diagram of an image processing apparatus 1 b inaccordance with a second embodiment of the invention.

FIG. 8 is a flow chart showing the image processing method of the imageprocessing apparatus 1 b.

FIG. 9 is a block diagram of an image processing apparatus 1 c inaccordance with a third embodiment of the invention.

FIG. 10 is a flow chart showing the image processing method of the imageprocessing apparatus 1 c.

FIG. 11 is a block diagram of an image processing apparatus 21 a inaccordance with a fourth embodiment of the invention.

FIG. 12 is a diagram for describing the area in which the dispersion iscalculated for a pixel of interest.

FIG. 13 is a diagram for describing a contracting process by averaging.

FIG. 14 is a diagram for describing a contracting process by smoothing.

FIG. 15 is a block diagram of an image processing apparatus 21 b inaccordance with a fifth embodiment of the invention.

FIG. 16 is a diagram for describing the method of calculating thecorrelation of peripheral pixels.

FIG. 17 is a flow chart showing the image processing method of the imageprocessing apparatus in accordance with the seventeenth to thetwenty-third inventions.

FIG. 18 is a flow chart showing the image processing method of the imageprocessing apparatus in accordance with the twenty-fourth invention.

FIG. 19 is a flow chart showing the image processing method of the imageprocessing apparatus in accordance with the twenty-fifth invention.

FIG. 20 is a block diagram of an image processing apparatus 31 a inaccordance with a seventh embodiment of the invention.

FIG. 21 is a diagram for describing a positional relationship detectingsection 34.

FIG. 22 is a graph for describing the mean for extracting a highbrightness component.

FIG. 23 is a block diagram of an image processing apparatus 31 b inaccordance with an eighth embodiment of the invention.

FIG. 24 is a graph for describing the operation of an image correctingsection 38.

FIG. 25 is a block diagram of an image processing apparatus 31 c inaccordance with a ninth embodiment of the invention.

FIG. 26 is a diagram for describing the operation of an image dividingsection 39.

FIG. 27 is a graph for describing the method of calculating a pixelvalue t2.

FIG. 28 is a block diagram of an image processing apparatus 50 inaccordance with a tenth embodiment of the invention.

FIG. 29 is a schematic diagram showing the configuration of a bookinputted to the image processing apparatus 50.

FIG. 30 is a diagram for describing the page-contour detecting operationof a page-contour detecting section 52.

FIG. 31 is a flow chart for describing the page-contour detectingtechnique of the page-contour detecting section 52.

FIG. 32 is a diagram for describing the page-contents region extractingoperation of a page-contents region extracting section 53.

FIG. 33 is a flow chart for describing the page-contents regionextracting technique of the page-contents region extracting section 53.

FIG. 34 is a diagram showing the situation of an image rotation.

FIG. 35 is a diagram showing the form of a template stored in a pagepositional information buffer 60.

BEST MODE FOR CARRYING OUT THE INVENTION

Now referring to the drawings, preferred embodiments of the inventionare described below.

(First Embodiment)

FIG. 1 is a block diagram of an image processing apparatus 1 a inaccordance with a first embodiment of the invention. The imageprocessing apparatus 1 a comprises an image inputting section 2, a coloranalyzing section 3, a mixing-ratio calculating section 4, an imageconverting section 5, and an image outputting section 6. The imageinputting section 2 reads an original image, for example, an image of acartoon journal, in a predetermined unit, such as a spread and a page,and inputs it to the color analyzing section 3. The color analyzingsection 3 analyzes the color used in the input image. The mixing-ratiocalculating section 4 determines the mixing-ratio r:g:b of therespective color components of red (R), green (G), and blue (B). Theimage converting section 5 mixes the color components of R, G and Bbased on the determined mixing-ratio, thereby converting the input imageinto a monochromatic image. The image outputting section 6 outputs theconverted image.

The image inputting section 2 is implemented by an image readingapparatus, such as a scanner, a copying machine, and a camera. It mayalso be implemented by an apparatus for reading an image from an medium,such as a CD-ROM (compact disk-read only memory), a hard disk, a floppydisk and a magneto-optical disk, which contains an image previously readfrom an original, as well as by a semiconductor memory.

The image outputting section 6 is implemented by an image displayingapparatus, such as a CRT (cathode ray tube) and an LCD (liquid crystaldisplay). It may also be an image printing apparatus such as a printer.Further, it may also be implemented by an apparatus for writing out animage on an medium, such as a CD-ROM, a hard disk, a floppy disk and amagneto-optical disk, as well as by a semiconductor memory.

The color analyzing section 3, the mixing-ratio calculating section 4and the image converting section 5 are implemented, for example, by acomputer and a software.

FIG. 2 is a flow chart showing the image processing method of the imageprocessing apparatus 1 a. The image inputting section 2 inputs anoriginal image in a predetermined unit (S1). The color analyzing section3 analyzes the color used in the input image (S2). The mixing-ratiocalculating section 4 determines the mixing-ratio r:g:b of therespective color components of R, G and B (S3). The image convertingsection 5 mixes the color components of R, G and B according to themixing-ratio, thereby converting the input image to a monochromaticimage (S4). The image outputting section 6 outputs the converted image(S5).

FIG. 3 is a flow chart for describing the color analyzing section 3. Thecolor analyzing section 3 converts the input image in a predeterminedunit into hue (H), saturation (S) and lightness (L) (S21). Theconversion of the input image into H, S and L can be carried out by awell known method (description is omitted). Then, the representative hueand the dispersion are obtained from the distribution of H and S (S22).Then, the presence or absence of a black pixel is determined from thehistogram of L (S23), thereby determining the color used in the image(S24).

FIG. 4 is a graph for describing the step S22. FIG. 4( a) is a graphhaving a horizontal axis indicating hue (H) and a vertical axisindicating the sum of saturation (S). The sum of S is defined by the sumof the values of S of all the pixels having an identical value of H. Itis shown as a histogram weighted by S in order to obtain the color usedin the image. Selecting the H giving the maximum of the sum of S, let itbe the representative hue H0. When the sum of S at H0 is less than orequal to a predetermined value, the input image is determined as anoriginally monochromatic image, and the representative hue H0 isdetermined as absent.

Using the relation H±2π=H, the sum of S is transformed so that the sumof S distributes in a range of H0±2π, thereby obtaining the dispersionV. When the dispersion V is greater than or equal to a predeterminedvalue, it is determined that a plurality of colors are used in the inputimage. Otherwise, it is determined that the representative hue H0 aloneis used.

FIG. 5 is a graph for describing the step S23. When a histogram oflightness (L) is prepared in 256 steps, black not being used decreasesthe low lightness pixels as shown in FIG. 5( a), and black being usedincreases the low lightness pixels as shown in FIG. 5( b). Accordingly,whether black is used or not can be determined depending on whether thedistribution of the pixels in a low lightness range is within apredetermined range or not.

FIG. 6 is a graph for describing the step S24. When the representativehue H0 is absent, the input image is determined as a monochromatic image(S241). When the dispersion V obtained in the step S22 is greater thanor equal to a predetermined value, it is determined that a plurality ofcolors are used in the input image (S242). Otherwise, the input image isdetermine as a monochromatic image in a color other than black if ablack pixel is absent, and the input image is determine as an image inblack plus another one color if a black pixel is present (S243).

Using the representative hue H0, the color used in the image isdetermined as follows. First, determine which of 0, π/3, 2π/3, π, 4π/3,5π/3 and 2π is the closest to H0. Here, H0 is assumed to be within therange of 0–2π. It is respectively determined as red (R) for 0 and 2π,yellow (Y) for π/3, green (G) for 2π/3, cyan (C) for π, blue (B) for4π/3 and magenta (M) for 5π/3.

The mixing-ratio calculating section 4 is described below. Themixing-ratio calculating section 4 determines the mixing-ratio r:g:b ofthe R, G and B components based on the analysis result of the coloranalyzing section 3. That is, it changes the mixing-ratiocorrespondingly to the color used in the image.

When the color analyzing section 3 determines as a monochromatic image,r=0, g=1 and b=0 are used, for example. In a monochromatic image, thereis no difference in R, G and B components, and hence, any of the r, gand b may be unity. Further, all of them may be an identical value.

When the color analyzing section 3 determines as plural colors, r=0.299,g=0.587 and b=0.114 are used similarly in an ordinarycolor/monochromatic conversion. Using such a mixing-ratio, a full-colorimage such as a photographic image can be converted to a naturallymonochromatic image.

When the color analyzing section 3 determines as a single color otherthan black, the mixing-ratio is assigned by the ratio of the colorcomponents of the complimentary color of the color used in the imagewhich is obtained and inputted by the color analyzing section 3. Here,the complimentary color is the color which becomes white when mixed withthat color. For example, when the color used in the input image is red(R), its complimentary color is cyan (C). Since the ratio of the R, Gand B components of cyan is 0:1:1, required mixing-ratio becomes r=0,g=0.5 and b=0.5. By mixing the R, G and B in such a mixing-ratio, theconversion to a monochromatic image is achieved with the highestcontrast of the image.

When the color analyzing section 3 determines as black plus one colorother than black, the mixing-ratio is assigned by the ratio of the colorcomponents of the complimentary color of the color used in the imageobtained by the color analyzing section 3, with adjustment by the ratioof the color components used in the image. For example, when the colorsused in the input image are red (R) and black, the ratio of the colorcomponents of red (R) is 1:0:0, and the ratio of the R, G and Bcomponents of cyan (C) which is the complimentary color of red is 0:1:1.In order to set the red between black and white for the distinctionbetween red and black, the ration of red is reduced to the half of theratio of cyan, and added. Therefore, required mixing-ratio r:g:b becomesr=0.2, g=0.4 and b=0.4. Here, the ratio between the color used and itscomplimentary color is set to 1:2, however, the ratio may be changedbased on the color used.

By being previously provided with a mixing-ratio table containing theoptimum mixing-ratio for each of the above-mentioned colors and colorcombinations, the mixing-ratio calculating section 4 may merely refer tothe mixing-ratio table. This avoids the necessity of calculating everytime, thereby speeding up the process.

Further, in the case where the image conversion is performed image byimage and that each of the images has a bias in color, the result of theimage conversion sometimes differs, which causes uncomfortableness inturning over the leaf of the electronic book viewed in a portableterminal such as a viewer. Thus, a plurality of images may be convertedas a whole. Specifically, the image inputting section 2 inputs aplurality of images, and the color analyzing section 3 analyzes thecolor from the result of the integration of the color used in each imagefrom among the plurality of images. The mixing-ratio calculating section4 calculates a mixing-ratio common to all the input images, and theimage converting section 5 converts all the images in the samemixing-ratio. Accordingly, images can be converted to monochromaticimages more stably.

Finally, the image converting section 5 converts each pixel into amonochromatic image M₀ based on the conversion formulaM ₀ =rR+gG+bB (where r+g+b=1),using the mixing-ratio r:g:b of the respective R, G and B colorcomponents determined by the mixing-ratio calculating section 4.(Second Embodiment)

FIG. 7 is a block diagram of an image processing apparatus 1 b inaccordance with a second embodiment of the invention. The imageprocessing apparatus 1 b comprises an image inputting section 2, amixing-ratio calculating section 4, an image converting section 5, animage outputting section 6 and a color specifying section 8. The imageinputting section 2, the mixing-ratio calculating section 4, the imageconverting section 5, and the image outputting section 6 are identicalto those of the above-mentioned image processing apparatus 1 a, andhence, the description is omitted. The color specifying section 8specifies the color used in the input image.

FIG. 8 is a flow chart showing the image processing method of the imageprocessing apparatus 1 b. The image inputting section 2 inputs an image(S1). The color specifying section specifies the color used in the inputimage (S6). The mixing-ratio calculating section 4 determines themixing-ratio r:g:b of the respective color components of R, G and B(S3). The image converting section 5 mixes the color components of R, Gand B according to the mixing-ratio, thereby converting the input imageto a monochromatic image (S4). The image outputting section 6 outputsthe image (S5).

The difference from FIG. 2 of the first embodiment is only the pointthat the step S2 is replaced by the step S6. In contrast to the imageprocessing apparatus 1 a automatically determining the color used in theinput image, the image processing apparatus 1 b semi-automaticallydetermines it by a user externally specifying it through the colorspecifying section 8 implemented by a mouse or a keyboard.

The color specifying section 8 is described below. The color specifyingsection 8 permits a user to select the kind of the image from the groupconsisting of monochromatic, plural colors, one color other than black,and black plus one color other than black. When one color other thanblack or black plus one color other than black is selected, the colorother than black is further selected from the group consisting of red(R), yellow (Y), green (G), cyan (C), blue (B) and magenta (M).Accordingly, by a user specifying the determination of the color, theimage can be determined more accurately to be converted to amonochromatic image.

(Third Embodiment)

FIG. 9 is a block diagram of an image processing apparatus 1 c inaccordance with a third embodiment of the invention. The imageprocessing apparatus 1 c comprises an image inputting section 2, animage converting section 5, an image outputting section 6 and amixing-ratio specifying section 9. The image inputting section 2, theimage converting section 5, and the image outputting section 6 areidentical to those of the above-mentioned image processing apparatus 1a, and hence, the description is omitted. The mixing-ratio specifyingsection 9 specifies the mixing-ratio r:g:b of the respective colorcomponents of R, G and B.

FIG. 8 is a flow chart showing the image processing method of the imageprocessing apparatus 1 b. The image inputting section 2 inputs an image(S1). The mixing-ratio specifying section 9 specifies the mixing-ratior:g:b of the respective color components of R, G and B (S7). The imageconverting section 5 mixes the color components of R, G and B accordingto the mixing-ratio, thereby converting the input image to amonochromatic image (S4). The image outputting section 6 outputs theimage (S5).

The difference from FIG. 2 of the first embodiment is only the pointthat the steps S2 and S3 are replaced by the step S7. In contrast to theimage processing apparatus 1 a automatically determining the color usedin the input image and automatically determining also the mixing-ratio,the image processing apparatus 1 c semi-automatically determines themixing-ratio by a user externally specifying the mixing-ratio of thecolor components through the mixing-ratio specifying section 9implemented by a mouse or a keyboard.

The mixing-ratio specifying section 9 is described below. Themixing-ratio specifying section 9 permits a user to select themixing-ratio r:g:b of the R, G and B components. Accordingly, the colorcomponents can be mixed in the ratio desired by a user, resulting in adesirable monochromatic image.

The processes shown in the first to the third embodiments areimplemented with a program. The program may be recorded on acomputer-readable recording medium, such as an optical disk and a floppydisk, to be used after being read out when necessary. An imageprocessing apparatus and an image processing method each for such aprocess are also included within the scope of the invention.

(Fourth Embodiment)

FIG. 11 is a block diagram of an image processing apparatus 21 a inaccordance with a fourth embodiment of the invention. The imageprocessing apparatus 21 a comprises an image inputting section 22, acharacter/line drawing region extracting section 23, a pseudo-densityregion extracting section 24, an image contracting section 25, and animage outputting section 26. The image inputting section 22 inputs animage. The character/line drawing region extracting section 23 extractsa character/line drawing region. The pseudo-density region extractingsection 24 extracts a pseudo-density region from the input image and theextracted results of the character/line drawing region extractingsection 23. The image contracting section 25 performs a contractingprocess by different methods respectively in the character/line drawingregion, the pseudo-density region and the region other than thecharacter/line drawing region and the pseudo-density region. The imageoutputting section 26 outputs the processed image.

The image inputting section 22 and the image outputting section 26 areimplemented similarly to the image inputting section 2 and the imageoutputting section 6, respectively.

The character/line drawing region extracting section 23 performs asmoothing process on the input image, and then performs edge extractionthereof. The smoothing process permits to extract the edge componentalone of a character and a line drawing after accurately eliminating thepseudo-density region even when the line drawing exists within apseudo-density region. In the smoothing process, a filter such as

$\begin{matrix}\begin{bmatrix}{1/25} & {1/25} & {1/25} & {1/25} & {1/25} \\{1/25} & {1/25} & {1/25} & {1/25} & {1/25} \\{1/25} & {1/25} & {1/25} & {1/25} & {1/25} \\{1/25} & {1/25} & {1/25} & {1/25} & {1/25} \\{1/25} & {1/25} & {1/25} & {1/25} & {1/25}\end{bmatrix} & (2)\end{matrix}$is used to simply average the pixel density in the periphery. Althoughthe size of the filter is 5×5 here, it may be changed depending on theresolution of the original image. Further, the filter used may be aGaussian filter and the like in which the central part and theperipheral part are differently weighted. In the edge extraction, twoedge extracting filters such as

$\begin{matrix}{{{vertical}\mspace{14mu}{{edge}\mspace{14mu}\begin{bmatrix}{- 1} & 0 & 1 \\{- 1} & 0 & 1 \\{- 1} & 0 & 1\end{bmatrix}}}{and}} & (3) \\{{horizontal}\mspace{14mu}{{edge}\mspace{14mu}\begin{bmatrix}1 & 1 & 1 \\0 & 0 & 0 \\{- 1} & {- 1} & {- 1}\end{bmatrix}}} & (4)\end{matrix}$are used to obtain the result of edge extraction which is defined by thesum of the absolute value of the output of each filter.

Using the result of the edge extraction, a pixel having a valueexceeding a predetermined threshold value is determined as acharacter/line drawing region. An isolated point and the like among thepixels having a value exceeding the predetermined threshold value can beexcluded from the character/line drawing region, considering it as anoise. A small region surrounded by edges can be included in a characterregion, because it can be considered as a complicated character area.

The pseudo-density region extracting section 24 is described below. Thepseudo-density region extracting section 24 calculates the dispersion ofthe peripheral pixels, and extracts, as a pseudo-density region, thepixel which is one of the pixels having a large dispersion and is notincluded in the character/line drawing region extracted by thecharacter/line drawing region extracting section 23. This depends on thefact that the pseudo-density region alone can be extracted by excludingthe character/line drawing regions from the pixels having a largedispersion, because the dispersion is large both in a pseudo-densityregion and in the edge portion of the character/line drawing region.

FIG. 12 is a diagram for describing the area in which the dispersion iscalculated for a pixel of interest. In FIG. 12, the shaded portion is apixel of interest, and the area surrounded by thick lines is the area tocalculate the dispersion. Although the dispersion for each pixel iscalculated in a 5×5 region in FIG. 12, the size of the region may bechanged depending on the resolution of the original image. Thedispersion for each pixel is obtained by

$\begin{matrix}\frac{\sum\limits_{i = 1}^{n}\left( {p_{i} - m} \right)^{2}}{n} & (5)\end{matrix}$Here, p_(i) is the pixel value, m is the average of the pixel densitywithin the region to survey the dispersion, and n is the number ofpixels included within the region to survey the dispersion.

The image contracting section 25 is described below. The imagecontracting section 25 contracts the pseudo-density region by asmoothing process, contracts the character/line drawing region bycontraction due to an averaging process and by a subsequent edgeenhancing process, and contracts the region other than thepseudo-density region and the character/line drawing region by theaveraging process alone. Accordingly, a moire is prevented in thepseudo-density region. In the character/line drawing region, unclearnessand blurring are prevented by the averaging process, and the clearnessis kept by the edge enhancing process. In region other than thepseudo-density region and the character/line drawing region, theperforming of the averaging process alone prevents unnecessary blurringby a smoothing process and the increase in noise by unnecessary edgeenhancement. In the case where a character/line drawing region and apseudo-density region are in the near vicinity, the averaging processand the edge enhancing process occurs in the near vicinity, whereby theimage quality extremely changes. Thus, the boundary pixels may be set asa region other than the pseudo-density region and the character/linedrawing region, whereby the image quality changes smoothly.

FIG. 13 is a diagram for describing the contracting process byaveraging. In FIG. 13, the small lattice, which is shown by brokenlines, is the lattice of the original image, and the large lattice shownby thick lines is the lattice of the contracted image. The contractedimage is on a scale of ½ of the original image for simplicity, however,the scale is not necessarily a reciprocal number of a whole number. Inthe case where the scale is not a reciprocal number of a whole number,the lattices of the original image and the contracted image does notcoincide. In that case, for example, a coordinate value may be roundedinto a whole number, or a pixel may be averaged with a weightcorresponding to the ratio of the areas common to the lattices. Thepixel value of the pixel Pm of interest shown in FIG. 13 is obtained byaveraging the shaded pixels.

FIG. 14 is a diagram for describing the contracting process bysmoothing. In FIG. 14, a small lattice which is shown by broken lines isa lattice of the original image, and the large lattice shown by thicklines is the lattice of the contracted image. The pixel value of thepixel Pb of interest shown in FIG. 14 is obtained by averaging theshaded pixels which extend into the rather wider area than the latticeof the contracted image. The area to perform the smoothing may bechanged depending on the contraction ratio and the resolution of theoriginal image.

The edge enhancement is performed using a filter such as

$\begin{matrix}\begin{bmatrix}{{- 1}/8} & {{- 1}/8} & {{- 1}/8} \\{{- 1}/8} & 2 & {{- 1}/8} \\{{- 1}/8} & {{- 1}/8} & {{- 1}/8}\end{bmatrix} & (6)\end{matrix}$Performing the edge enhancing process on the contracted image permits toclarify the dullness of the image due to the contraction by theaveraging process.(Fifth Embodiment)

FIG. 15 is a block diagram of an image processing apparatus 21 b inaccordance with a fifth embodiment of the invention. The imageprocessing apparatus 21 b comprises an image inputting section 22, acharacter/line drawing region extracting section 23, an imagecontracting section 25, an image outputting section 26 and apseudo-density region extracting section 27. The image inputting section22 inputs an image. The character/line drawing region extracting section23 extracts a character/line drawing region. The pseudo-density regionextracting section 27 extracts a pseudo-density region from the inputimage and the results of the character/line drawing region extractingsection 23. The image contracting section 25 performs a contractingprocess by different methods respectively in the character/line drawingregion, the pseudo-density region and the region other than thecharacter/line drawing region and the pseudo-density region. The imageoutputting section 26 outputs the processed image. The image inputtingsection 22, the character/line drawing region extracting section 23, theimage contracting section 25 and the image outputting section 26 areimplemented similarly to those of the image processing apparatus 21 a,and hence, the description is omitted.

The pseudo-density region extracting section 27 calculates thecorrelation of the periphery of a pixel of interest and extracts, as apseudo-density region, a pixel which is one of the pixels having a lowcorrelation and is not included in the character/line drawing regionextracted by the character/line drawing region extracting section 23.Since the pseudo-density region has a low correlation with theperipheral pixels, and since the edge portion has a large correlation inany of the vertical, horizontal and oblique directions, the probabilityis decreased that a character and a line drawing portion are classifiedinto a pseudo-density region.

FIG. 16 is a diagram for describing the method of calculating thecorrelation of peripheral pixels. First, a reference region which is aregion containing a pixel of interest is defined. Comparative regions Bi(i=1, 2, 3, 4) each of which is a shifted region of the reference regionA in any of the four of vertical, horizontal and oblique directions:(+1, 0), (+1, +1), (0, +1), (−1, −1) in (x, y) direction are defined.The value of correlation C for the pixel of interest is obtained by

$\begin{matrix}{C = {{Min}\left( \frac{{A - {Bi}}}{n} \right)}} & (7)\end{matrix}$

Where |A−Bi| represents the total sum of the absolute values of thedifferences between the corresponding pixels in the regions A and B, nis the number of the pixels in the region, and Min indicates the minimumwithin i=1, . . . , 4. The image processing apparatus 21 b calculatesthe value of correlation by the difference between the regions, however,the other methods may be used. A larger value of correlation C indicatesa lower correlation, and a smaller value indicates a higher correlation.Since vertical, horizontal, and oblique lines has a large correlation inany of the above-mentioned four directions, they can be excluded from apseudo-density region at an early stage, thereby permitting to extract apseudo-density region more accurately.

The pseudo-density region extracting section 27 extracts, as apseudo-density region, a pixel which is one of the pixels having anabove-mentioned value of correlation greater than a predetermined value(having a low correlation) and is not extracted by the character/linedrawing region extracting section 23. Accordingly, pseudo-densityregions alone can be accurately extracted.

(Sixth Embodiment)

The image processing apparatus in accordance with a sixth embodiment isthe image processing apparatus 21 a of the fourth embodiment or theimage processing apparatus 21 b of the fifth embodiment with thepseudo-density region extracting sections 24, 27 changed. The othercomponents are implemented similarly to those of the image processingapparatuses 21 a, 21 b of the fourth and the fifth embodiments, andhence, the description is omitted. The pseudo-density region extractingsection of the image processing apparatus in accordance with the sixthembodiment detects the edge detection of an image and extracts, as apseudo-density region, a pixel which is one of the pixels having a largeedge value and is not included in the character/line drawing regionextracted by the character/line drawing region extracting section 23.This depends on the fact that the pseudo-density region alone can beextracted by excluding the character/line drawing regions from thepixels having a large edge value, because the output of a edge detectingfilter is large both in a pseudo-density region and in the edge portionof the character/line drawing region.

FIG. 17 is a flow chart showing the image processing method of the imageprocessing apparatus in accordance with the seventeenth to thetwenty-third inventions. A step S31 is a process module for inputting animage, that is, a process module for reading an image from an imageinputting apparatus, such as a scanner, or a storage medium into amemory of the image contracting apparatus. A step 32 is a process modulefor extracting a character/line drawing region, and extracts thecharacter/line drawing region by edge extraction after performing asmoothing process on the input image. A step S33 is a process module forextracting a pseudo-density region and extracts the pseudo-densityregion by the method described in the fourth, the fifth and the sixthembodiments. A step S34 is a process module for performing the imagecontracting process described in the fourth, the fifth and the sixthembodiments on the character/line drawing region and the pseudo-densityregion extracted by the process modules of the steps S32, S33. A stepS35 is a process module for outputting the image contracted by the stepS34 into the image outputting section 26.

FIG. 18 is a flow chart showing the image processing method of the imageprocessing apparatus in accordance with the twenty-fourth invention.Steps S40–S43 are process modules for performing the same processes asthe steps S31–S34 of FIG. 17, and hence, the description is omitted. Astep S44 detects an edge of a pseudo-density region already contracted.Since the pseudo-density region is processed by the smoothing process,the edge value normally should not be large. However, in the case of theoccurrence of a moire, an edge should be detected.

A step S45 is a process module for performing the smoothing process onlyon the pixels (for example, 60) having an edge value which is detectedby the step S44 and is greater than or equal to a predetermined value.Accordingly, it is characterized by performing the smoothing process onthe contracted image. A step S46 is a process module for outputting theimage, and performs the same process as the step S35.

FIG. 19 is a flow chart showing the image processing method of the imageprocessing apparatus in accordance with the twenty-fifth invention.Steps S50–S53 are process modules for performing the same processes asthe steps S31–S34 of FIG. 17 and the steps S40–S43 of FIG. 18. A stepS54 detects an edge of a pseudo-density region already contracted. Sincethe pseudo-density region is processed by the smoothing process, theedge value normally should not be large. However, in the case of theoccurrence of a moire, an edge should be detected.

In the case where there exists a pixel (for example, 60) having an edgevalue which is detected by the step S54 and is greater than or equal toa predetermined value, a step S55 interrupts the contracting process,indicating the occurrence of a moire (step S57). It may merely generatethe warning, in stead. In the case where there is not a pixel having anedge value greater than or equal to a predetermined value, the step S55outputs the image and terminates (step S56).

(Seventh Embodiment)

FIG. 20 is a block diagram of an image processing apparatus 31 a inaccordance with a seventh embodiment of the invention. The imageprocessing apparatus 31 a comprises an image inputting section 32, animage reversing sections 33 a, 33 b, a positional relationship detectingsection 34, an image correcting section 35 and an image outputtingsection 36. The image inputting section 32 inputs the front and backimages of an original. The first image reversing section 33 a reversesthe right and left of the back image alone. The positional relationshipdetecting section 34 detects the positional relationship between thefront and back images. The image correcting section 35 eliminates aghost image from each of the front and back images by a calculatingprocess on the front and back images depending on the positionalrelationship of the front and back images. The second image reversingsection 33 b recovers the orientation of the back image by revertingagain the right and left of the back image. The image outputting section36 outputs the corrected front and back images. The output can be usedas an electronic book. The image inputting section 32 and the imageoutputting section 36 can be implemented similarly to those of the imageinputting section 2 and the image outputting section 6, and hence, thedescription is omitted.

The first image reversing section 33 a reverses the right and left ofthe back image. The front image may be reversed. Since the image on theopposite side which causes a ghost image is in the reversed orientationof right and left, the back image is reversed before the calculatingprocess by the positional relationship detecting section 34 and theimage correcting section 35. After the correcting process, the correctedback image is reversed again by the second image reversing section 33 b,thereby recovering the normal orientation. The image reversing sections33 a, 33 b may have an identical configuration.

FIG. 21 is a diagram for describing the positional relationshipdetecting section 34. The positional relationship detecting section 34is implement, for example, by a block matching section. A referenceregion F having a size of (m×n) is defined on the front image side, anda comparative region G having a size of (s×t) larger than the referenceregion F is defined on the back image side. The comparative region G iscompared with the reference region F thereby to obtain a region havingthe highest similarity to the reference region F. In other words, letthe top left point of the reference region F be (0, 0), and any top leftpoint of the comparative region G be (u, v). Then, the comparisonbetween an arbitrary point (k, l) of the reference region F and thecorresponding point (k+u, l+v) of the comparative region G is carriedout by the following Formula (8).

$\begin{matrix}{{d\left( {u,v} \right)} = {\sum\limits_{k = 0}^{n - 1}{\sum\limits_{l = 0}^{m - 1}{{{G\left( {{k + u},{l + v}} \right)} - {F\left( {k,l} \right)}}}}}} & (8)\end{matrix}$

The region of the comparative region G which locates the position suchthat the point d(u, v) of the Formula (8) becomes minimum is determinedas having the highest similarity to the reference region F, therebydetecting the coincidence of the comparative region G with the referenceregion F.

Knowing the positional relationship of the correspondence between thefront and back images, the amount of the shift (0x, 0y) of the backimage with respect to the front image can be obtained from thedifference between the positions of the comparative region G and thereference region F.

In the embodiment, the block matching is carried out in one referenceregion F, thereby obtaining the amount of parallel displacement only.However, the block matching may be carried out in two or more referenceregions F, thereby detecting also the amount of rotation.

Since a ghost image has very low brightness and contrast in comparisonwith the original image, there is an anxiety that the direct comparisonbetween the front and back images during the block matching can resultsin a wrong detection of the position due to the strong influence of theoriginal image on each side. Thus, the invention comprises highbrightness component extracting means for previously extracting the highbrightness component alone before the positional relationship detectingsection 34, thereby performing the block matching using the highbrightness component alone, thereby achieving more accurate detection ofthe position.

FIG. 22 is a graph for describing the mean for extracting a highbrightness component. In the graph of FIG. 22, the horizontal axisindicates the input pixel value, and the vertical axis indicates theoutput pixel value. In the graph of FIG. 22, the pixel value representsthe brightness and has a value of 0–255. A value nearer to −0 isdetermined as a lower brightness (black), and a value nearer to 255 as ahigher brightness (white). However, this relationship maybe reversed,the value is not necessarily a whole number, and the range is notnecessarily 0–255.

The means for extracting a high brightness component converts the pixelvalue alone of the input image having a pixel value greater than orequal to Lt into a value of 0–255, thereby cutting off a low brightnesscomponent to extract the high brightness component alone. The pixelvalue Lt is previously set to a value lower than the brightness of aghost image component, and depends on the transmittance of an originalsheet, the sensitivity characteristics of a scanner, and the like.

The operation of the image correcting section 35 is described below withreference to the following Formulae (9)–(12). The following a and brepresent the actually printed front and back images, respectively. TheA and B represent the front and back images, respectively, which areread by the image inputting section 32 and includes a ghost image. Forthe simplicity of description, a, b, A and B represent the respectivepixel values corresponding to each other in an identical position.However, in practice, the corresponding pixels are determined by thepositional relationship detecting section 34 considering a paralleldisplacement and a rotation.

$\begin{matrix}{A = {a - {r\left( {255 - b} \right)}}} & (9) \\{B = {b - {r\left( {255 - a} \right)}}} & (10) \\{a = \frac{A + {r\left( {255 - B} \right)} - {255\mspace{11mu} r^{2}}}{1 - r^{2}}} & (11) \\{b = \frac{B + {r\left( {255 - A} \right)} - {255\mspace{11mu} r^{2}}}{1 - r^{2}}} & (12)\end{matrix}$

The r represents the transmittance of the medium, such as an originalsheet, on which the image is printed. It can be obtained by substitutingthe known or measured values for A, a and b in Formula (9).

The Formulae (9) and (10) are solved, resulting in Formulae (11) and(12). That is, by the calculating process of the photographed frontimage A and back image B, the actual front image a and back image b canbe recovered eliminating a ghost image. The image correcting section 35performs the calculation of the above-mentioned Formulae (11) and (12)and outputs the front image a and the back image b.

The image processing apparatus 31 a can be implemented by acomputer-readable recording medium, such as a floppy, a ROM and a CD,which records a program describing the process steps of the imagereversing sections 33 a, 33 b, the positional relationship detectingsection 34 and the image correcting section 35 described above.

(Eighth Embodiment)

FIG. 23 is a block diagram of an image processing apparatus 31 b inaccordance with an eighth embodiment of the invention. The imageprocessing apparatus 31 b comprises an image inputting section 32, animage outputting section 36, an edge detecting section 37 and an imagecorrecting section 38. The image inputting section 32 inputs an image.The edge detecting section 37 detects an edge. The image correctingsection 38 eliminates a ghost image by raising the brightness of a pixelother than the edge and a low brightness region. The image outputtingsection 36 outputs the corrected image. The image inputting section 32and the image outputting section 36 of the image processing apparatus 31b are implemented similarly to those of the image inputting section 32and the image outputting section 36 of the image processing apparatus 31a, and hence, the description is omitted.

In the edge detection by the edge detecting section 37, two edgedetection filters such as Equations (13) and (14) are used, and theresult of the edge detection is defined by the sum of the absolute valueof the outputs of the respective filters. Using the result of the edgedetection, a pixel having a value exceeding a predetermined thresholdvalue is determined as an edge.

$\begin{matrix}{{vertical}\mspace{14mu}{{edge}\mspace{14mu}\begin{bmatrix}{- 1} & 0 & 1 \\{- 1} & 0 & 1 \\{- 1} & 0 & 1\end{bmatrix}}} & (13) \\{{horizontal}\mspace{14mu}{{edge}\mspace{14mu}\begin{bmatrix}1 & 1 & 1 \\0 & 0 & 0 \\{- 1} & {- 1} & {- 1}\end{bmatrix}}} & (14)\end{matrix}$

FIG. 24 is a graph for describing the operation of the image correctingsection 38. Let the pixel value in the periphery of a ghost imagecomponent the input pixel value of which is previously known be t2, anda pixel value appropriately smaller than the pixel value t2 be t1. Then,the image correcting section 38 corrects a ghost image by changing therelationship (inclination) to the output pixel value between the pixelvalue t1 and the pixel value t2 to correct the output pixel value so asto saturate at the pixel value t2 and above. The pixel values t1, t2depend on the transmittance of an original sheet, the sensitivitycharacteristics of a scanner, and the like. Here, since there is noinfluence on the low brightness component having a pixel value smallerthan or equal to t1, the whitening-out of a black portion can beprevented. Further, the brightness correction is not carried out in theedge portion detected by the edge detecting section 37, whereby the highbrightness portion in the contour of a character is conserved, wherebythe unclearness of a character can be prevented.

The image processing apparatus 31 b can be implemented by acomputer-readable recording medium, such as a floppy, a ROM and a CD,which records a program describing the process steps of the edgedetecting section 37 and the image correcting section 38 describedabove.

(Ninth Embodiment)

FIG. 25 is a block diagram of an image processing apparatus 31 c inaccordance with a ninth embodiment of the invention. The imageprocessing apparatus 31 c comprises an image inputting section 32, animage outputting section 36, an edge detecting section 37, an imagedividing section 39 and an image correcting section 40. The imageinputting section 32 inputs an image. The edge detecting section 37detects an edge. The image dividing section 39 divides the image at thedetected edge and low brightness component. The image correcting section40 eliminates a ghost image by raising the brightness of a portionhaving a high average brightness within the divided region. The imageoutputting section 36 outputs the corrected image. The image inputtingsection 32, the image outputting section 36 and the edge detectingsection 37 of the image processing apparatus 31 c are implementedsimilarly to those of the image inputting section 32, the imageoutputting section 36 and the edge detecting section 37 of the imageprocessing apparatus 31 b, and hence, the description is omitted.

The image dividing section 39 divides an image region based on the edgeand the pixel value exceeding a predetermined threshold value detectedby the edge detecting section 37. For example, as shown in FIG. 26, theimage dividing section 39 divides the image into regions 1–5. The region1 is a surrounded region having a character, the region 2 is a baseregion, the region 3 is a black halftone region, the region 4 is a thinhalftone region, and the region 5 is a thick halftone region. Assumethat the regions 1–5 does not include a black pixel or an edge portion,such as a character. Further assume that a ghost image having the samebrightness as the region.4 exists in the regions 1 and 2. The regions3–5 using a halftone screen have a low average brightness, however, theregions 1 and 2 have a high average brightness because of the whitebackground. Accordingly, by performing a brightness correction in thehigh average brightness regions 1 and 2 alone, the ghost image in theregions 1 and 2 can be eliminated with the halftone in the region 4preserved even when the regions 1 and 2 have a ghost image having thesame brightness as the region 4. Accordingly, a black region in an imageis excluded from the region to correct a ghost image because of theunremarkableness of a ghost image.

The image correcting section 40 calculates the average brightness in aregion divided by the image dividing section 39, and corrects thebrightness of a high brightness portion by the similar method of FIG. 24described in the eighth embodiment only when the brightness is greaterthan or equal to a predetermined value, thereby eliminating a ghostimage. As described above, by performing the ghost-image correction inhigh average brightness regions alone, even a halftone portion havingthe same brightness as a ghost image can be prevented from thewhitening-out of the uniform region surrounded by lines. Further, sinceblack regions and edge portions are excluded, the unclearness of acharacter and the whitening-out of a black portion are prevented.

The image correcting section 40 automatically calculates the pixel valuet2 of the FIG. 24 from the distribution of the pixel values in a highbrightness region. FIG. 27 is a graph for describing the method ofcalculating the pixel value t2. First, the histogram of the pixel valuesof a high brightness region is prepared, and pixel values tmin and tmaxare defined. When all the pixels of the high brightness region exist inthe right of tmax, set t2=tmax. When all the pixels of the highbrightness region exist in the left of tmin, set t2=tmin. When the pixelhaving the minimum exists between tmin and tmax, let the value be thepixel value t2. Since a ghost image portion is darker than theperipheral region, the ghost image can be eliminated by detecting thepixel value and by correcting it so as to be white.

The image processing apparatus 31 c can be implemented by acomputer-readable recording medium, such as a floppy, a ROM and a CD,which records a program describing the process steps of the edgedetecting section 37, image dividing section 39 and the image correctingsection 40 described above.

(Tenth Embodiment)

FIG. 28 is a block diagram of an image processing apparatus 50 inaccordance with a tenth embodiment of the invention. In the imageprocessing apparatus 50, an image inputting section 51 reads the imagedata page by page from an original of a book separated into pages. Apage contour detecting section 52, a page contents region extractingsection 53, an inclination correcting section 54, a page positioncorrecting section 55 and a page information processing section 57perform the processes described later using various buffers. The imageoutputting section 56 outputs the image data in which the alignmentbetween the pages is corrected.

FIG. 29 is a schematic diagram showing the configuration of a novel bookas an example of a book inputted to the image processing apparatus 50.In FIG. 29, the book inputted to the image processing apparatus 50consists of a title (cover) page, contents table pages, main-text pages(even pages and odd pages), index pages and a back cover page. Amajority of the pages are main-text pages each having a header region(shows a page number in the present example), a footer region (shows achapter number in the example) and a contents region (main-text region)in a fixed position. The image processing apparatus 50 aligns the pagesusing such a feature.

The processes by respective sections of the image processing apparatus50 are described below. The image inputting section 51 reads a binary ormulti-valued image from a scanner and the like, and saves it in an inputimage buffer 58. This image may be a monochromatic image or a colorimage. The orientation of the image input is roughly correct, and theinput is carried out in the sequence of the pages from the first or thelast. In the case of the reading by a scanner and the like, it ispreferable that the input region of the scanner is larger than the sizeof a page, considering the case of the inclined input due to the errorof an automatic feeder, if used. This situation is assumed in thefollowing description. When a page is larger than the input region ofthe scanner, it can be inputted in divided pieces and reconstructed inthe input image buffer 58.

The page contour detecting section 52 is described below. Since the pageis smaller than the input region of the scanner, the image inputted bythe image inputting section 51 consists of an actual page region of thebook and a background region behind it. The page contour detectingsection 52 discriminates between the background region and the pageregion form the input image, thereby extracts the contour of the pageregion. Here, in the case where a book is used after being separatedinto pages, a page sometimes has the inclined or torn edge correspondingto the back portion. In that case, the contour is not an accuraterectangle, but is assumed to be approximated by a rectangle.

The methods for detecting the contour of a page region include a methodin which an edge portion is detected from the image thereby detectingthe fact that each angle of the rectangle is 90 degree from the edgepoint, a method in which an ensemble of edge points having a largechange in brightness is extracted thereby obtaining a straight line fromthe ensemble thereby extracting the contour and the like.

An example of a technique of detecting the contour of a page region isdescribed below with reference to FIGS. 30 and 31. FIG. 30 is a diagramfor describing this technique. FIG. 31 is a flow chart for describingthis technique. Since the contour of a page region is a rectangle, amethod of detecting the four straight lines in the outermost location ofthe image is described herein.

First, description is made for the case of detecting a contour line inthe left side of the page region shown in FIG. 30( a) (S61, S62). A lineto scan is selected firstly (S63). The top row is selected here becausethe scan is carried out horizontally. Since the scan is carried out fromleft to right, the left end is set to the initial value (i=0) (S64).Scanning the image sequentially, the brightness of each point isacquired from the input image buffer 58 (S56). Determination is carriedout whether the point is an edge point or not (S66). The method of thedetermination is to calculate the first-order differential in thehorizontal direction. An example of this is the method using a Sorbelfilter. When the point is determined as an edge point, the coordinate ofthe point is stored (S69), the scanning of the line is terminated, andthe next line is selected. Such a scan is carried out through all thelines to the bottom row (S69). When the point is determined not as anedge point, it is determined whether an edge of the image or not (S67).When it is determined as an edge of the image, it proceeds to S69. Whenit is determined not as an edge of the image, i=i+1 is set (S68), and itreturns to S65. As a result, an ensemble of the coordinate of the edgepoints is obtained. A majority of these accumulate on a straight line.Thus, the straight line is calculated. This is typically carried out bythe half transformation (S71). The process described above is carriedout in each of the four directions (corresponding to D=0–3 in the flowof FIG. 31) of the input image (FIG. 30( b)) (S72), thereby obtainingfour straight lines (S73), thereby detecting these straight lines as thecontour of the page region (FIG. 30( c)) (S74).

The page contents region extracting section 53 is described below withreference to FIGS. 32 and 33. The page contents region extractingsection 53 extracts a page contents region from the image within thecontour of the page region obtained by the page contour detectingsection 52. FIG. 32 is a diagram for describing this technique. FIG. 33is a flow chart for describing this technique.

As shown in FIG. 32( a), scanning the image in the sequence of the lines(S61, S62), an edge point is extracted. This is by the same method asthe page contour detecting section 52. This edge point may be an edge ofa character, a section line in a figure or table, or an edge of aballoon of a cartoon. However, an edge point of a character region is tobe obtained in the present example. In the case of a character stringand the like, the obtained edge point ensemble does not exists on astraight line. Thus, obtaining the lines (straight lines) circumscribingsuch an edge ensemble (FIG. 32( b)), the straight lines are set as theboundary of the page contents region (FIG. 32( c)) herein.

The method of obtaining the circumscribing line is described again withreference to FIG. 33. First, an edge point ensemble is obtained (S75).Selecting two points from the edge point ensemble, the equation of thestraight line passing through the two points is obtained (S76). Theequation of the straight line is(y2−y1)x−(x2−x1)y−(x1y2−x2y1)=0with the coordinate of the two points (x1, y1) and (x2, y2).

It is determined on which side of the straight line another edge pointnot selected exists (S77). The determining equation isF(x, y)=(y2−y1)x+(x2−x1)y−x1y2+x2y1where the point (x, y) locates on the origin side when F(x, y)<0 andoutside when F(x, y)>0.

When all the points locate on the same side, the straight line is acircumscribing line. Otherwise, another two points are selected(S78–80). Repeating for all the combination, an circumscribing line canbe obtained definitely.

By carrying out the process described above in each of the four scanningdirections (corresponding to D=0–3 in the flow of FIG. 33) (S72), fourstraight lines is obtained (S73), whereby the page contents region canbe extracted as a figure surrounded by the circumscribing lines (FIG.32( c)) (S74).

The inclination correcting section 54 is described below. Theinclination correcting section 54 performs the process of rotating withrespect to the reference coordinate axes based on the figure of thecircumscribing lines extracted by the page contents region extractingsection 53, thereby correcting the inclination of the input image. Theprocess is carried out on all the pages of the book.

FIG. 34 is a diagram showing the situation of the image rotation. Thecenter of the rotating conversion is set to the coordinate (cx, cy) ofan corner of the page contents region. In a rotation of the input imageby θ, the coordinate of a point on the input image is (x, y) and thecoordinate of a point on the corrected image after the conversion is(x′, y′). The conversion formula of this rotation is

$\begin{pmatrix}x^{\prime} \\y^{\prime}\end{pmatrix} = {{\begin{pmatrix}{{\cos(\theta)} - {\sin(\theta)}} \\{{\sin(\theta)}\mspace{14mu}{\cos(\theta)}}\end{pmatrix}\begin{pmatrix}{x - {cx}} \\{y - {cy}}\end{pmatrix}} + \begin{pmatrix}{cx} \\{cy}\end{pmatrix}}$

In the rotating process, this formula is applied to each pixel on theinput image buffer 58, thereby assigning a brightness or a color to theconverted coordinate on a corrected image buffer 59.

The page information processing section 57 is described below. The pageinformation processing section 57 determines whether the input image ofa book is a main-text page or the other page. The method of thedetermination is a method in which the size or the shape of the figureof the circumscribing lines extracted by the page contents regionextracting section 53 is compared and it is determined as a main-textpage when the size or the shape is within a predetermined range. Analternative method is a method in which consecutive pages are determinedas the main-text pages during the consecutive pages having an almostconstant size of the rectangle of the circumscribing lines continue.This method depends on the fact that the input is carried out in thesequence of the pages and that the main-text pages have an almostconstant size of the rectangle of the circumscribing lines detected bythe page contents region extracting section 53. Another alternativemethod is a method in which the first page of and the last page (sheetnumbers) of the main-text pages are previously specified externally.This method also depends on the fact that the input is carried out inthe sequence of the pages. Further, when determining as a main-textpage, the page information processing section 57 stores the positionalinformation of the page contents region in the rectangle of thecircumscribing lines into a page positional information buffer 60, anduses it as a template for aligning the main-text images and, hence, thepages. When the page positional information buffer 60 already containsthe positional information of the template indicating a main-text pageused as a reference for the page alignment, it proceeds to the pageposition correcting section 55.

The page position correcting section 55 is described below. Although theinclination of an input image is corrected, the position of the pagecontents region of the main-text page varies image by image because ofmechanical deviation and the like during the reading. If the intactmain-text pages are outputted as an electronic book without positioncorrection, the positional variation occurs during the viewing of thecontents of the main-text pages in a viewer and the like, which causesuncomfortableness. Thus, the page position correcting section 55corrects, using a parallel displacement, the position of the images ofthe main-text pages the inclination of which is corrected, so as tocoincide with the positional information of the template indicating amain-text page stored in the page positional information buffer 60. As aresult, the image outputting section 56 can provide the image data inwhich the main-text pages are aligned. In other words, once the templateis obtained from a page determined as a main-text page, the subsequentmain-text pages are aligned with the reference to the template.

The above-mentioned template includes all of the header region, thefooter region and the contents region of a main-text page. However, eachregion may be separated as shown in FIG. 35( a), and stored into thepage positional information buffer 60 in the form shown in FIG. 35( b).Such a separation simplifies character recognition, key word extraction,and the like, thereby simplifying the structuring of a document.

In the above-mentioned description, all the processes are automated.However, the processes of the page contour detecting section 52, thepage contents region extracting section 53, and the inclinationcorrecting section 54 may be carried out manually.

(Eleventh Embodiment)

The images of a book is ordinarily inputted automatically andsequentially. Accordingly, whether the next input is a main-text page orthe other is not known at the time of the input. Further, even in thecase of a main-text page, the size of the page contents region differs,for example, in the last page of a chapter. Thus, even in the case wherea page is determined as a main-text page, when the positionalinformation of the page contents region of the newly input image differsfrom the positional information of the template in the page positionalinformation buffer 60 (when the positional information is out of apredetermined range), an error is concluded, an error bit is writteninto an error buffer (not shown) page by page, and warning data isgenerated and maintained. Accordingly, a user can easily recognize thepages the error of which is to be corrected by a manual process, bywatching the warning data from the error buffer through display means(not shown) after the completion of the automatic process of the wholebook.

The series of the above-mentioned processes can implemented with aprogram. The program may be recorded on a computer-readable recordingmedium, such as an optical disk and a floppy disk, to be used afterbeing read out when necessary.

The invention may be embodied in other specific forms without departingfrom the spirit or essential characteristics thereof. The embodimentsare therefore to be considered in all respects as illustrative and notrestrictive, the scope of the invention being indicated by the appendedclaims rather than by the foregoing description and all changes whichcome within the meaning and the range of equivalency of the claims aretherefore intended to be embraced therein.

INDUSTRIAL APPLICABILITY

In accordance with the first invention, it is possible to produce amonochromatic image in such a manner that the color analyzing meansanalyzes the color used within the image inputted from the imageinputting means, the mixing-ratio calculating means calculates themixing-ratio of color components such as red, green and blue based onthe analyzed color, and the converting means mixes color componentsaccording to the calculated mixing-ratio. By such an image processingapparatus, the color of the input image is automatically determined anda monochromatic image can be produced.

In accordance with the second invention, it is possible to produce amonochromatic image in such a manner that the color analyzing meansanalyzes colors used within a plurality of images inputted from theimage inputting means, the mixing-ratio calculating means calculates themixing-ratios of color components which are common to the plurality ofimages, based on the analyzed color, and the converting means mixescolor components according to the calculated mixing-ratios. By such animage processing apparatus, the colors of the plurality of input imagesare automatically determined and monochromatic images can be produced.Further, since the color determination is carried out in the pluralityof input images, the color can be determined more accurately.Furthermore, since monochromatic images are produced in the sameconditions for the plurality of input images, the images can be producedstably.

In accordance with the third invention, it is possible to produce amonochromatic image in such a manner that an image is inputted from theimage inputting means, the color specifying means specifies the colorused within the image, the mixing-ratio calculating means calculates themixing-ratio of color components based on the specified color, theconverting means mixes the color components according to the calculatedmixing-ratio. By such an image processing apparatus, a monochromaticimage with higher accuracy can be produced by specifying the color usedwithin the input image by a user.

In accordance with the fourth invention, an image is inputted from theimage inputting means, the mixing-ratio specifying means specifies themixing-ratio of color components for the image, and the converting meansmixes the color components based on the specified mixing-ratio. By suchan image processing apparatus, a desired monochromatic image can beproduced by specifying the mixing-ratio of a color component by a user.

In accordance with the fifth invention, data is inputted, the color usedwithin the image is analyzed, the mixing-ratio of color components iscalculated based on the analyzed color, and color components are mixedaccording to the calculated mixing-ratio. By such an image processingmethod, the color of the input image is automatically determined and amonochromatic image can be produced.

In accordance with the sixth invention, the color used within the imageis analyzed based on the distribution of the hue, saturation andlightness of the input image. By such an image processing method, thecolor of the input image is automatically determined and a monochromaticimage can be produced.

In accordance with the seventh invention, the data of a plurality oforiginal images is inputted, the colors used within the images areanalyzed, the mixing-ratios of color components, common to the pluralityof images, are calculated based on the analyzed colors, and colorcomponents are mixed according to the calculated mixing-ratio. By suchan image processing method, the colors of the plurality of input imagesare automatically determined and respective monochromatic images can beproduced. Further, since colors are determined from the plurality ofinput images, the colors can be determined more accurately. Furthermore,since monochromatic images are produced in the same conditions for theplurality of input images, the images can be produced stably.

In accordance with the eighth invention, the color used within the inputimage is specified, the mixing-ratio of color components is calculatedbased on the color, and color components are mixed according to thecalculated mixing-ratio. By such an image processing method, amonochromatic image with higher accuracy can be produced by specifyingthe color used within the input image by a user.

In accordance with the ninth invention, the mixing-ratio is calculatedby referring to the mixing-ratio table. By such an image processingmethod, the color of the input image is automatically determined and amonochromatic image can be produced. Further, by calculating themixing-ratio with reference to the mixing-ratio table, optimummixing-ratio is rapidly obtained for each color used within the image.Thus, more optimum monochromatic image can be produced at high speed.

In accordance with the tenth invention, the mixing-ratio is calculatedaccording to the color components ratio of the complimentary color ofthe color used within the input image. By such an image processingmethod, the color of the input image is automatically determined and amonochromatic image with high contrast can be produced.

In accordance with the eleventh invention, the mixing-ratio iscalculated based on the color component ratio of the complimentary colorof the color used within the input image and the color component ratioused within the input image. By such an image processing method, thecolor of the input image is automatically determined, and ahigh-contrast monochromatic image in which the color used in the inputimage is easily discriminated from black can be produced.

In accordance with the twelfth invention, the mixing-ratio of the colorcomponents of the input image is specified, and color components aremixed according to the mixing-ratio. By such an image processing method,a desired monochromatic image can be produced by specifying themixing-ratio of a color component by a user.

In accordance with the thirteenth invention, a computer analyzes a colorused within an input image, calculates the mixing-ratio of colorcomponents based on the color, and mixes the color components accordingto the mixing-ratio. Accordingly, a medium can be provided which recordsan image processing program such that the color of the input image isautomatically determined and a monochromatic image is produced.

In accordance with the fourteenth invention, a computer analyzes thecolors used within a plurality of input images, calculates themixing-ratios of color components, common to the plurality of inputimages, based on the colors, and mixes the color components according tothe mixing-ratios. Accordingly, a medium can be provided on which animage processing program is recorded such that the colors of theplurality of input images are automatically determined and monochromaticimages are produced.

In accordance with the fifteenth invention, a medium can be provided onwhich an image processing program of a computer for producing amonochromatic image with higher accuracy by specifying a color usedwithin an input image by a user is recorded, the image processingprogram comprising the steps of specifying a color used within an inputimage, calculating a mixing-ratio of color components based on thecolor, mixing color components according to the mixing-ratio, andproducing a monochromatic image with higher accuracy.

In accordance with the sixteenth invention, a computer specifies amixing-ratio for an input image, and mixes the color componentsaccording to the specified mixing-ratio. Accordingly, a medium can beprovided on which an image processing program is recorded such that adesired monochromatic image is produced by a user specifying themixing-ratio of a color component.

In accordance with the seventeenth invention, the image inputting meansinputs an image, from the image the character/line drawing regionextracting means extracts a character/line drawing region and thepseudo-density region extracting means extracts a pseudo-density region,the image contracting means contracts the image using mutually differentmethods respectively in the pseudo-density region, the character/linedrawing region and the region other than the pseudo-density region andthe character/line drawing region, and the image outputting meansoutputs the contracted image. Using such an image processing apparatus,by dividing the input image into three regions, i.e., pseudo-densityregion, character/line drawing region and the other region, the imagecan be contracted with a moire being suppressed in the pseudo-densityregion, the image can be clearly contracted in the character/linedrawing region, and the image can be properly contracted in the otherregion.

In accordance with the eighteenth invention, the image contracting meanscontracts the image by a smoothing process in the pseudo-density region,contracts the image by an averaging process and a subsequent edgeenhancing process in the character/line drawing region, and contractsthe image by an averaging process in the other region, and the imageoutputting means outputs the contracted image. By such an imageprocessing apparatus, the image can be contracted with a moire beingsuppressed in the pseudo-density region, the image can be clearlycontracted in the character/line drawing region, and the image can beproperly contracted in the other region.

In accordance with the nineteenth invention, from the input image thecharacter/line drawing region extracting means extracts a character/linedrawing region, and thereafter the pseudo-density region extractingmeans extracts a pseudo-density region. By such an image processingapparatus, a character/line drawing region is firstly extract from theinput image, and a pseudo-density region is then extracted. Therefore,the character/line drawing region can be accurately extracted withoutbeing affected from the pseudo-density region, even when it existswithin the pseudo-density region.

In accordance with the twentieth invention, the character/line drawingregion is extracted by the edge extraction of the input image after thesmoothing process thereof. By such an image processing apparatus, acharacter/line drawing region is firstly extract from the input image asmentioned above, and a pseudo-density region is then extracted.Therefore, the character/line drawing region can be accurately extractedwithout being affected from the pseudo-density region, even when itexists within the pseudo-density region.

In accordance with the twenty-first invention, the pseudo-density regionis extracted by calculating the dispersion of the peripheral pixelsaround each pixel of the input image and by extracting the pixel whichis one of the pixels having a large dispersion and exists in the regionwhich is not extracted as a character/line drawing region. Using such animage processing apparatus, by calculating the dispersion of peripheralpixels and by extracting, as a pseudo-density region, the pixel is oneof the pixels having a large dispersion and exists in the region whichis not extracted as a character/line drawing region, the character/linedrawing region is eliminated, whereby the pseudo-density region alonecan be extracted accurately.

In accordance with the twenty-second invention, the pseudo-densityregion is extracted by calculating the correlation of the peripheralpixels around each pixel of the input image and by extracting the pixelwhich is one of the pixels having a low correlation and exists in theregion which is not extracted as a character/line drawing region. Usingsuch an image processing apparatus, by calculating the correlation ofperipheral pixels and by extracting, as a pseudo-density region, thepixel which is one of the pixels having a low correlation and exists inthe region which is not extracted as a character/line drawing region,the character/line drawing region is eliminated more securely, wherebythe pseudo-density region alone can be extracted accurately.

In accordance with the twenty-third invention, the pseudo-density regionis extracted by detecting an edge region of the input image and byextracting the region which is one of the edge regions and is notextracted as a character/line drawing region. By such an imageprocessing apparatus, the edge filter is simple, and the pseudo-densityregion can be extracted faster.

In accordance with the twenty-fourth invention, the edge detection iscarried out in the pseudo-density region and the smoothing process isrepeated for a region having a density greater than or equal to apredetermined value. By such an image processing apparatus, the imagecan be precisely contracted with a moire being suppressed more securelyin the pseudo-density region.

In accordance with the twenty-fifth invention, the edge detection iscarried out in the pseudo-density region and the contracting process isinterrupted for a region having a density greater than or equal to apredetermined value. By such an image processing apparatus, the normalcontracting process can be continued with an unnecessary contractingprocess being avoided.

In accordance with the twenty-sixth invention, an image is inputted, acharacter/line drawing region is extracted from the image, and apseudo-density region is extracted, the image is contracted usingmutually different methods respectively in the pseudo-density region,the character/line drawing region and the other region, and the image isthen outputted. By such an image processing method, the image can becontracted with a moire being suppressed in the pseudo-density region,the image can be clearly contracted in the character/line drawingregion, and the image can be properly contracted in the other region.

In accordance with the twenty-seventh invention, a computer extracts acharacter/line drawing region from the image, extracts a pseudo-densityregion, contracts the image using mutually different methodsrespectively in the pseudo-density region, the character/line drawingregion and the other region, and outputs it. Accordingly, a medium canbe provided which records an image processing program such that theimage is contracted with a moire being suppressed in the pseudo-densityregion, the image is clearly contracted in the character/line drawingregion, and the image is properly contracted in the other region.

In accordance with the twenty-eighth invention, front and back imagesare inputted from the image inputting means; after one of the images isreversed by the image reversing means, the positional relationshipbetween the front and back images is detected by the positionalrelationship detecting means; and the image is corrected to be free froma ghost image by the image correcting means using the positionalrelationship, and then outputted by the image outputting means. By suchan image processing apparatus, the input image can be outputted withouta ghost image.

In accordance with the twenty-ninth invention, the positionalrelationship between the front and back images is detected by extractingthe high brightness component alone of the front and back images and byperforming the block matching of the high brightness component. By suchan image processing apparatus, the positional relationship can bedetected more precisely, and the input image can be outputted moresecurely without a ghost image.

In accordance with the thirtieth invention, the image inputting meansinputs an image, the edge detecting means detects an edge of the image,the image correcting means corrects the image to eliminate a ghost imageof the image by raising the brightness of high brightness pixels otherthan the edge of the image outputted from the edge detecting means, andthe image outputting means outputs the image. By such an imageprocessing apparatus, the input image can be outputted with theunclearness of a character being prevented and without a ghost image.

In accordance with the thirty-first invention, the image inputting meansinputs an image, the edge detecting means detects an edge of the image,the image dividing means divides the image based on the edge and lowbrightness pixels of the image outputted from the edge detecting means,the image correcting means corrects the image to eliminate a ghost imageof the image by calculating the average brightness within a dividedregion and by raising the brightness of the high brightness regionalone, and the image outputting means outputs the image. By such animage processing apparatus, the input image can be outputted with thewhitening-out of a halftone section being prevented and without a ghostimage.

In accordance with the thirty-second invention, the image in a highbrightness region is corrected by obtaining the representativebrightness from the pixels having a brightness within a predeterminedrange and by raising the brightness of the region with the referencingto the representative brightness. By such an image processing apparatus,the input image can be outputted without a ghost image, free from theinfluence of the difference in the transmittance depending on the paperquality.

In accordance with the thirty-third invention, front and back images areinputted, after one of the images is reversed, the positionalrelationship between the front and back images is detected, and theimage is corrected to be free from a ghost image using the positionalrelationship, and then outputted. By such an image processing method,the input image can be outputted without a ghost image.

In accordance with the thirty-fourth invention, an image is inputted,and an edge, of the image is detected, and the image is corrected to befree from a ghost image of the image by raising the brightness of highbrightness pixels other than the edge of the image outputted from theedge detection, and then outputted. By such an image processing method,the input image can be outputted with the unclearness of a characterbeing prevented and without a ghost image.

In accordance with the thirty-fifth invention, an image is inputted, andan edge of the image is detected, the image is divided based on the edgeand the low brightness pixels of the image outputted from the edgedetection, and the image is corrected to be free from a ghost image ofthe image by calculating the average brightness within a divided regionand by raising the brightness of the high brightness region alone, andthen outputted. By such an image processing method, the input image canbe outputted with the whitening-out of a halftone section beingprevented and without a ghost image.

In accordance with the thirty-sixth invention, a computer reverses oneof inputted front and back images, after that, detects the positionalrelationship between the front and back images, corrects the image toeliminate a ghost image using the positional relationship, and outputsit. Accordingly, a medium can be provided which records an imageprocessing program such that the input image is outputted without aghost image.

In accordance with the thirty-seventh invention, a computer detects anedge of an inputted image, corrects the image to eliminate a ghost imageof the image by raising the brightness of high brightness pixels otherthan the edge of the image outputted from the edge detection, andoutputs it. Accordingly, a medium can be provided which records an imageprocessing program such that the input image is outputted with theunclearness of a character being prevented and without a ghost image.

In accordance with the thirty-eighth invention, a computer detects anedge of an inputted image, divides the image based on the edge and thelow brightness pixels of the image outputted from the edge detection,corrects the image to eliminate a ghost image of the image bycalculating the average brightness within a divided region and byraising the brightness of the high brightness region alone, and outputsit. Accordingly, a medium can be provided which records an imageprocessing program such that the input image is outputted with thewhitening-out of a halftone section being prevented and without a ghostimage.

In accordance with the thirty-ninth invention, the image inputting meansinputs an image page by page. The image determining means determines apredetermined image from among the images. The template acquiring meansacquires a template from the determined image. The image correctingmeans corrects the position between the images based on the template,thereby aligning the images of consecutive pages. By such an imageprocessing apparatus, the alignment between desired consecutive imagesfrom among the images inputted page by page can be carried out in ashort time.

In accordance with the fortieth invention, the image inputting meansinputs an image page by page of a book, the image determining meansdetermines a predetermined main-text image from among the images, thetemplate acquiring means acquires a template from the determined image,and the image correcting means corrects the position between main-textimages based on the template, thereby aligning the main-text images ofconsecutive pages. By such an image processing apparatus, the alignmentbetween main-text images from among the main-text images inputted pageby page can be carried out in a short time. Thus, the contents of anelectronic book can be prepared in a short term. Further, since theposition of the main-text images is aligned when the electronic book isviewed in a viewer, uncomfortableness to a user can be eliminated.

In accordance with the forty-first invention, a predetermined image isdetermined from among the images inputted page by page, a template isacquired from the determined image, and the position between the imagesis corrected based on the template, thereby aligning the images ofconsecutive pages. By such an image processing method, the alignmentbetween desired consecutive images from among the images inputted pageby page can be carried out in a short time.

In accordance with the forty-second invention, the template is acquiredas the positional information of the rectangle defined by thecircumscribing lines obtained from the ensemble of the edge pointsacquired by scanning the input image. By such an image processingmethod, since the template is acquired using the circumscribing lines,an accurate template can be obtained, thereby improving the precision ofthe alignment.

In accordance with the forty-third invention, a predetermined image isdetermined from among the input images, and warning data is generated inthe case where the positional information of the input image and thepositional information of the template are out of a predetermined range.By such an image processing method, failure in the alignment between theimages can be detected, and hence, there is convenience in revisionduring or after the authoring.

In accordance with the forty-fourth invention, a predetermined main-textimage is determined from among the images inputted page by page of abook, a template is acquired from the determined image, and the positionbetween main-text images is corrected based on the template, therebyaligning the main-text images of consecutive pages. By such an imageprocessing method, the alignment between main-text images from among themain-text images inputted page by page can be carried out in a shorttime. Thus, the contents of an electronic book can be prepared in ashort term. Further, since the position of the main-text images isaligned when the electronic book is viewed in a viewer,uncomfortableness to a user can be eliminated.

In accordance with the forty-fifth invention, a computer determines apredetermined image from among the images inputted page by page,acquires a template from the determined image, and corrects the positionbetween the images based on the template, thereby aligning the images ofthe consecutive pages. Accordingly, a medium can be provided whichrecords an image processing program such that the alignment betweendesired consecutive images from among the images inputted page by pageis carried out in a short time.

In accordance with the forty-sixth invention, a computer determines apredetermined main-text image from among the images inputted page bypage of a book, acquires a template from the determined image, andcorrects the position between main-text images based on the template,thereby aligning the main-text images of consecutive pages. Accordingly,a medium can be provided which records an image processing program suchthat the alignment between main-text images from among the main-textimages inputted page by page is carried out in a short time.

1. An image processing apparatus comprising: image inputting means forinputting front and back images of an original; image reversing meansfor reversing one of the front and back images; positional relationshipdetecting means for detecting the positional relationship between thefront image reversed by the image reversing means and the back imagefrom the image inputting means or the positional relationship betweenthe back image reversed by the image reversing means and the front imagefrom the image inputting means; image correcting means for correctingthe image to eliminate a ghost image of the image using the positionalrelationship between the front and back images obtained from thepositional relationship detecting means; and image outputting means foroutputting the image.
 2. The image processing apparatus of claim 1,wherein the positional relationship detecting means detects thepositional relationship between the front and back images by extractingthe high brightness component alone of the front and back images and byperforming the block matching of the high brightness component.
 3. Animage processing method comprising: an image reversing step of reversingone of front and back images of an original; a positional relationshipdetecting step of detecting a positional relationship between thereversed one and the other of the front and back images; and an imagecorrecting step of correcting the other one to eliminate a ghost imageof the reversed one using a result of the positional relationshipdetection.
 4. A medium on which an image processing program is recorded,the image processing program being for causing a computer to execute animage reversing step of reversing one of front and back images; apositional relationship detecting step of detecting a positionalrelationship between the reversed one and the other of the front andback images; and an image correcting step of correcting the image toeliminate a ghost image from the other using a result of the positionalrelationship detection.